Viewing: ChangeLog
18-01-2021 Whamcloud, Inc.
* version 2.14.0
* Support for networks:
socklnd - any kernel supported by Lustre
o2iblnd - OFED from any kernels supported by Lustre
o2iblnd - MLNX_OFED 4.5-1.0.1.0, MLNX_OFED 4.6-1.0.1.1
MLNX_OFED 4.7-1.0.0.1, MLNX_OFED 4.7-3.2.9.0
MLNX_OFED 4.9-0.1.7.0, MLNX_OFED 4.9-2.2.4.0
MLNX_OFED 5.0-2.1.8.0, MLNX_OFED 5.1-0.6.6.0
MLNX_OFED 5.1-2.3.7.1, MLNX_OFED 5.1-2.5.8.0
MLNX_OFED 5.2-1.0.4.0
* Features:
MR Routing: This feature aligns the LNET Multi-Rail behavior
with routing. A gateway now is viewed as a Multi-Rail
capable node. When a route is added only one entry
per gateway should be used. That route entry should
be added using a reachable nid of the gateway.
The multi-rail selection algorithm is then run when
sending to the gateway to select the best interface
to send to. The gateway aliveness is now kept via
the health mechanism.
-------------------------------------------------------------------------------
11-05-2018 Whamcloud, Inc.
* version 2.12.0
* Support for networks:
socklnd - any kernel supported by Lustre
o2iblnd - OFED from any kernels supported by Lustre
o2ilbnd - MLNX_OFED 3.4-2.0.0.0, MLNX_OFED 4.0-1.0.1.0
MLNX_OFED 4.0-2.0.0.1, MLNX_OFED 4.1-1.0.2.0
MLNX_OFED 4.2-1.0.0.0, MLNX_OFED 4.2-1.2.0.0
MLNX_OFED 4.3-1.0.1.0. MLNX_OFED 4.4-1.0.0.0
MLNX_OFED 4.4-2.0.7.0
* Features:
LNet Health: Keep track of network interface health and select
healthiest interface.
TBD Intel Corporation
* version 2.2.0
* Support for networks:
socklnd - any kernel supported by Lustre,
o2iblnd - OFED 1.1, 1.2.0, 1.2.5, 1.3, and 1.4.1
mxlnd - MX 1.2.10 or later
ptllnd - Portals 3.3 / UNICOS/lc 1.5.x, 2.0.x
-------------------------------------------------------------------------------
09-30-2011 Whamcloud, Inc.
* version 2.1.0
* Support for networks:
socklnd - any kernel supported by Lustre,
o2iblnd - OFED 1.1, 1.2.0, 1.2.5, 1.3, and 1.4.1
* Available but unsupported:
mxlnd - MX 1.2.10 or later
ptllnd - Portals 3.3 / UNICOS/lc 1.5.x, 2.0.x
-------------------------------------------------------------------------------
2010-07-15 Oracle, Inc.
* version 2.0.0
* Support for networks:
socklnd - any kernel supported by Lustre,
qswlnd - Qsnet kernel modules 5.20 and later,
openiblnd - IbGold 1.8.2,
o2iblnd - OFED 1.1, 1.2.0, 1.2.5, 1.3, and 1.4.1
viblnd - Voltaire ibhost 3.4.5 and later,
ciblnd - Topspin 3.2.0,
iiblnd - Infiniserv 3.3 + PathBits patch,
gmlnd - GM 2.1.22 and later,
mxlnd - MX 1.2.10 or later,
ptllnd - Portals 3.3 / UNICOS/lc 1.5.x, 2.0.x
Severity : minor
Bugzilla : 21459
Description: should update lp_alive for non-router peers
Severity : enhancement
Bugzilla : 15332
Description: LNet router shuffler.
Severity : enhancement
Bugzilla : 15332
Description: LNet fine grain routing support.
Severity : normal
Bugzilla : 20171
Description: router checker stops working when system wall clock goes backward
Details : use monotonic timing source instead of system wall clock time.
Severity : enhancement
Bugzilla : 18460
Description: avoid asymmetrical router failures
Severity : enhancement
Bugzilla : 19735
Description: multiple-instance support for kptllnd
Severity : normal
Bugzilla : 20897
Description: ksocknal_close_conn_locked connection race
Details : A race was possible when ksocknal_create_conn calls
ksocknal_close_conn_locked for already closed conn.
Severity : normal
Bugzilla : 18102
Description: router_proc.c is rewritten to use sysctl-interface for parameters
residing in /proc/sys/lnet
Severity : enhancement
Bugzilla : 13065
Description: port router pinger to userspace
Severity : normal
Bugzilla : 17546
Description: kptllnd HELLO protocol deadlock
Details : kptllnd HELLO protocol doesn't run to completion in finite time
Severity : normal
Bugzilla : 18075
Description: LNet selftest fixes and enhancements
Severity : enhancement
Bugzilla : 19156
Description: allow a test node to be a member of multiple test groups
Severity : enhancement
Bugzilla : 18654
Description: MXLND: eliminate hosts file, use arp for peer nic_id resolution
Details : an update from the upstream developer Scott Atchley.
Severity : enhancement
Bugzilla : 15332
Description: add a new LND optiion to control peer buffer credits on routers
Severity : normal
Bugzilla : 18844
Description: Fixing deadlock in usocklnd
Details : A deadlock was possible in usocklnd due to race condition while
tearing connection down. The problem resulted from erroneous
assumption that lnet_finalize() could have been called holding
some lnd-level locks.
Severity : major
Bugzilla : 13621, 15983
Description: Protocol V2 of o2iblnd
Details : o2iblnd V2 has several new features:
. map-on-demand: map-on-demand is disabled by default, it can
be enabled by using modparam "map_on_demand=@value@", @value@
should >= 0 and < 256, 0 will disable map-on-demand, any other
valid value will enable map-on-demand.
Oi2blnd will create FMR or physical MR for RDMA if fragments of
RD > @value@.
Enable map-on-demand will take less memory for new connection,
but a little more CPU for RDMA.
. iWARP : to support iWARP, please enable map-on-demand, 32 and 64
are recommanded value. iWARP will probably fail for value >=128.
. OOB NOOP message: to resolve deadlock on router.
. tunable peer_credits_hiw: (high water to return credits),
default value of peer_credits_hiw equals to (peer_credits -1),
user can change it between peer_credits/2 and (peer_credits - 1).
Lower value is recommended for high latency network.
. tunable message queue size: it always equals to peer_credits,
higher value is recommended for high latency network.
. It's compatible with earlier version of o2iblnd
Severity : normal
Bugzilla : 18414
Description: Fixing 'running out of ports' issue
Details : Add a delay before next reconnect attempt in ksocklnd in
the case of lost race. Limit the frequency of query-requests
in lnet. Improved handling of 'dead peer' notifications in
lnet.
Severity : normal
Bugzilla : 16034
Description: Change ptllnd timeout and watchdog timers
Details : Add ptltrace_on_nal_failed and bump ptllnd timeout to match
Portals wire timeout.
Severity : normal
Bugzilla : 16186
Description: One down Lustre FS hangs ALL mounted Lustre filesystems
Details : Shared routing enhancements - peer health detection.
Severity : enhancement
Bugzilla : 14132
Description: acceptor.c cleanup
Details : Code duplication in acceptor.c for the cases of kernel and
user-space removed. User-space libcfs tcpip primitives
uniformed to have prototypes similar to kernel ones. Minor
cosmetic changes in usocklnd to use cfs_socket_t as
representation of socket.
Severity : minor
Bugzilla : 11245
Description: IB path MTU mistakenly set to 1st path MTU when ib_mtu is off
Details : See comment 46 in bug 11245 for details - it's indeed a bug
introduced by the original 11245 fix.
Severity : minor
Bugzilla : 15984
Description: uptllnd credit overflow fix
Details : kptl_msg_t::ptlm_credits could be overflown by uptllnd since
it is only a __u8.
Severity : major
Bugzilla : 14634
Description: socklnd protocol version 3
Details : With current protocol V2, connections on router can be
blocked and can't receive any incoming messages when there is no
more router buffer, so ZC-ACK can't be handled (LNet message
can't be finalized) and will cause deadlock on router.
Protocol V3 has a dedicated connection for emergency messages
like ZC-ACK to router, messages on this dedicated connection
don't need any credit so will never be blocked. Also, V3 can send
keepalive ping in specified period for router healthy checking.
-------------------------------------------------------------------------------
12-31-2008 Sun Microsystems, Inc.
* version 1.8.0
* Support for networks:
socklnd - any kernel supported by Lustre,
qswlnd - Qsnet kernel modules 5.20 and later,
openiblnd - IbGold 1.8.2,
o2iblnd - OFED 1.1, 1.2.0, 1.2.5, and 1.3
viblnd - Voltaire ibhost 3.4.5 and later,
ciblnd - Topspin 3.2.0,
iiblnd - Infiniserv 3.3 + PathBits patch,
gmlnd - GM 2.1.22 and later,
mxlnd - MX 1.2.1 or later,
ptllnd - Portals 3.3 / UNICOS/lc 1.5.x, 2.0.x
Severity : major
Bugzilla : 15983
Description: workaround for OOM from o2iblnd
Details : OFED needs allocate big chunk of memory for QP while creating
connection for o2iblnd, OOM can happen if no such a contiguous
memory chunk.
QP size is decided by concurrent_sends and max_fragments of
o2iblnd, now we permit user to specify smaller value for
concurrent_sends of o2iblnd(i.e: concurrent_sends=7), which
will decrease memory block size required by creating QP.
Severity : major
Bugzilla : 15093
Description: Support Zerocopy receive of Chelsio device
Details : Chelsio driver can support zerocopy for iov[1] if it's
contiguous and large enough.
Severity : normal
Bugzilla : 13490
Description: fix credit flow deadlock in uptllnd
Severity : normal
Bugzilla : 16308
Description: finalize network operation in reasonable time
Details : conf-sanity test_32a couldn't stop ost and mds because it
tried to access non-existent peer and tcp connect took
quite long before timing out.
Severity : major
Bugzilla : 16338
Description: Continuous recovery on 33 of 413 nodes after lustre oss failure
Details : Lost reference on conn prevents peer from being destroyed, which
could prevent new peer creation if peer count has reached upper
limit.
Severity : normal
Bugzilla : 16102
Description: LNET Selftest results in Soft lockup on OSS CPU
Details : only hits when 8 or more o2ib clients involved and a session is
torn down with 'lst end_session' without preceeding 'lst stop'.
Severity : minor
Bugzilla : 16321
Description: concurrent_sends in IB LNDs should not be changeable at run time
Details : concurrent_sends in IB LNDs should not be changeable at run time
Severity : normal
Bugzilla : 15272
Description: ptl_send_rpc hits LASSERT when ptl_send_buf fails
Details : only hits under out-of-memory situations
-------------------------------------------------------------------------------
2009-02-07 Sun Microsystems, Inc.
* version 1.6.7
* Support for networks:
socklnd - any kernel supported by Lustre,
qswlnd - Qsnet kernel modules 5.20 and later,
openiblnd - IbGold 1.8.2,
o2iblnd - OFED 1.1, 1.2.0, 1.2.5, and 1.3
viblnd - Voltaire ibhost 3.4.5 and later,
ciblnd - Topspin 3.2.0,
iiblnd - Infiniserv 3.3 + PathBits patch,
gmlnd - GM 2.1.22 and later,
mxlnd - MX 1.2.1 or later,
ptllnd - Portals 3.3 / UNICOS/lc 1.5.x, 2.0.x
Severity : major
Bugzilla : 15983
Description: workaround for OOM from o2iblnd
Details : OFED needs allocate big chunk of memory for QP while creating
connection for o2iblnd, OOM can happen if no such a contiguous
memory chunk.
QP size is decided by concurrent_sends and max_fragments of
o2iblnd, now we permit user to specify smaller value for
concurrent_sends of o2iblnd(i.e: concurrent_sends=7), which
will decrease memory block size required by creating QP.
Severity : major
Bugzilla : 15093
Description: Support Zerocopy receive of Chelsio device
Details : Chelsio driver can support zerocopy for iov[1] if it's
contiguous and large enough.
Severity : normal
Bugzilla : 13490
Description: fix credit flow deadlock in uptllnd
Severity : normal
Bugzilla : 16308
Description: finalize network operation in reasonable time
Details : conf-sanity test_32a couldn't stop ost and mds because it
tried to access non-existent peer and tcp connect took
quite long before timing out.
Severity : major
Bugzilla : 16338
Description: Continuous recovery on 33 of 413 nodes after lustre oss failure
Details : Lost reference on conn prevents peer from being destroyed, which
could prevent new peer creation if peer count has reached upper
limit.
Severity : normal
Bugzilla : 16102
Description: LNET Selftest results in Soft lockup on OSS CPU
Details : only hits when 8 or more o2ib clients involved and a session is
torn down with 'lst end_session' without preceeding 'lst stop'.
Severity : minor
Bugzilla : 16321
Description: concurrent_sends in IB LNDs should not be changeable at run time
Details : concurrent_sends in IB LNDs should not be changeable at run time
-------------------------------------------------------------------------------
11-03-2008 Sun Microsystems, Inc.
* version 1.6.6
* Support for networks:
socklnd - any kernel supported by Lustre,
qswlnd - Qsnet kernel modules 5.20 and later,
openiblnd - IbGold 1.8.2,
o2iblnd - OFED 1.1, 1.2.0, 1.2.5, and 1.3
viblnd - Voltaire ibhost 3.4.5 and later,
ciblnd - Topspin 3.2.0,
iiblnd - Infiniserv 3.3 + PathBits patch,
gmlnd - GM 2.1.22 and later,
mxlnd - MX 1.2.1 or later,
ptllnd - Portals 3.3 / UNICOS/lc 1.5.x, 2.0.x
Severity : normal
Bugzilla : 15272
Description: ptl_send_rpc hits LASSERT when ptl_send_buf fails
Details : only hits under out-of-memory situations
-------------------------------------------------------------------------------
04-26-2008 Sun Microsystems, Inc.
* version 1.6.5
* Support for networks:
socklnd - any kernel supported by Lustre,
qswlnd - Qsnet kernel modules 5.20 and later,
openiblnd - IbGold 1.8.2,
o2iblnd - OFED 1.1 and 1.2.0, 1.2.5
viblnd - Voltaire ibhost 3.4.5 and later,
ciblnd - Topspin 3.2.0,
iiblnd - Infiniserv 3.3 + PathBits patch,
gmlnd - GM 2.1.22 and later,
mxlnd - MX 1.2.1 or later,
ptllnd - Portals 3.3 / UNICOS/lc 1.5.x, 2.0.x
Severity : normal
Bugzilla : 14322
Description: excessive debug information removed
Details : excessive debug information removed
Severity : major
Bugzilla : 15712
Description: ksocknal_create_conn() hit ASSERTION during connection race
Details : ksocknal_create_conn() hit ASSERTION during connection race
Severity : major
Bugzilla : 13983
Description: ksocknal_send_hello() hit ASSERTION while connecting race
Details : ksocknal_send_hello() hit ASSERTION while connecting race
Severity : major
Bugzilla : 14425
Description: o2iblnd/ptllnd credit deadlock in a routed config.
Details : o2iblnd/ptllnd credit deadlock in a routed config.
Severity : normal
Bugzilla : 14956
Description: High load after starting lnet
Details : gmlnd should sleep in rx thread in interruptible way. Otherwise,
uptime utility reports high load that looks confusingly.
Severity : normal
Bugzilla : 14838
Description: ksocklnd fails to establish connection if accept_port is high
Details : PID remapping must not be done for active (outgoing) connections
--------------------------------------------------------------------------------
2008-01-11 Sun Microsystems, Inc.
* version 1.4.12
* Support for networks:
socklnd - any kernel supported by Lustre,
qswlnd - Qsnet kernel modules 5.20 and later,
openiblnd - IbGold 1.8.2,
o2iblnd - OFED 1.1 and 1.2.0, 1.2.5
viblnd - Voltaire ibhost 3.4.5 and later,
ciblnd - Topspin 3.2.0,
iiblnd - Infiniserv 3.3 + PathBits patch,
gmlnd - GM 2.1.22 and later,
mxlnd - MX 1.2.1 or later,
ptllnd - Portals 3.3 / UNICOS/lc 1.5.x, 2.0.x
Severity : normal
Bugzilla : 14387
Description: liblustre network error
Details : liblustre clients should understand LNET_ACCEPT_PORT environment
variable even if they don't start lnet acceptor.
Severity : normal
Bugzilla : 14300
Description: Strange message from lnet (Ignoring prediction from the future)
Details : Incorrect calculation of peer's last_alive value in ksocklnd
--------------------------------------------------------------------------------
2007-12-07 Cluster File Systems, Inc. <info@clusterfs.com>
* version 1.6.4
* Support for networks:
socklnd - any kernel supported by Lustre,
qswlnd - Qsnet kernel modules 5.20 and later,
openiblnd - IbGold 1.8.2,
o2iblnd - OFED 1.1 and 1.2.0, 1.2.5.
viblnd - Voltaire ibhost 3.4.5 and later,
ciblnd - Topspin 3.2.0,
iiblnd - Infiniserv 3.3 + PathBits patch,
gmlnd - GM 2.1.22 and later,
mxlnd - MX 1.2.1 or later,
ptllnd - Portals 3.3 / UNICOS/lc 1.5.x, 2.0.x
Severity : normal
Bugzilla : 14238
Description: ASSERTION(me == md->md_me) failed in lnet_match_md()
Severity : normal
Bugzilla : 12494
Description: increase send queue size for ciblnd/openiblnd
Severity : normal
Bugzilla : 12302
Description: new userspace socklnd
Details : Old userspace tcpnal that resided in lnet/ulnds/socklnd replaced
with new one - usocklnd.
Severity : enhancement
Bugzilla : 11686
Description: Console message flood
Details : Make cdls ratelimiting more tunable by adding several tunable in
procfs /proc/sys/lnet/console_{min,max}_delay_centisecs and
/proc/sys/lnet/console_backoff.
--------------------------------------------------------------------------------
2007-09-27 Cluster File Systems, Inc. <info@clusterfs.com>
* version 1.6.3
* Support for networks:
socklnd - any kernel supported by Lustre,
qswlnd - Qsnet kernel modules 5.20 and later,
openiblnd - IbGold 1.8.2,
o2iblnd - OFED 1.1 and 1.2,
viblnd - Voltaire ibhost 3.4.5 and later,
ciblnd - Topspin 3.2.0,
iiblnd - Infiniserv 3.3 + PathBits patch,
gmlnd - GM 2.1.22 and later,
mxlnd - MX 1.2.1 or later,
ptllnd - Portals 3.3 / UNICOS/lc 1.5.x, 2.0.x
Severity : normal
Bugzilla : 12782
Description: /proc/sys/lnet has non-sysctl entries
Details : Updating dump_kernel/daemon_file/debug_mb to use sysctl variables
Severity : major
Bugzilla : 13236
Description: TOE Kernel panic by ksocklnd
Details : offloaded sockets provide their own implementation of sendpage,
can't call tcp_sendpage() directly
Severity : normal
Bugzilla : 10778
Description: kibnal_shutdown() doesn't finish; lconf --cleanup hangs
Details : races between lnd_shutdown and peer creation prevent
lnd_shutdown from finishing.
Severity : normal
Bugzilla : 13279
Description: open files rlimit 1024 reached while liblustre testing
Details : ulnds/socklnd must close open socket after unsuccessful
'say hello' attempt.
Severity : major
Bugzilla : 13482
Description: build error
Details : fix typos in gmlnd, ptllnd and viblnd
--------------------------------------------------------------------------------
2007-07-30 Cluster File Systems, Inc. <info@clusterfs.com>
* version 1.6.1
* Support for networks:
socklnd - kernels up to 2.6.16,
qswlnd - Qsnet kernel modules 5.20 and later,
openiblnd - IbGold 1.8.2,
o2iblnd - OFED 1.1 and 1.2
viblnd - Voltaire ibhost 3.4.5 and later,
ciblnd - Topspin 3.2.0,
iiblnd - Infiniserv 3.3 + PathBits patch,
gmlnd - GM 2.1.22 and later,
mxlnd - MX 1.2.1 or later,
ptllnd - Portals 3.3 / UNICOS/lc 1.5.x, 2.0.x
--------------------------------------------------------------------------------
2007-06-21 Cluster File Systems, Inc. <info@clusterfs.com>
* version 1.4.11
* Support for networks:
socklnd - kernels up to 2.6.16,
qswlnd - Qsnet kernel modules 5.20 and later,
openiblnd - IbGold 1.8.2,
o2iblnd - OFED 1.1
viblnd - Voltaire ibhost 3.4.5 and later,
ciblnd - Topspin 3.2.0,
iiblnd - Infiniserv 3.3 + PathBits patch,
gmlnd - GM 2.1.22 and later,
mxlnd - MX 1.2.1 or later,
ptllnd - Portals 3.3 / UNICOS/lc 1.5.x, 2.0.x
Severity : minor
Bugzilla : 13288
Description: Initialize cpumask before use
Severity : major
Bugzilla : 12014
Description: ASSERTION failures when upgrading to the patchless zero-copy
socklnd
Details : This bug affects "rolling upgrades", causing an inconsistent
protocol version negotiation and subsequent assertion failure
during rolling upgrades after the first wave of upgrades.
Severity : minor
Bugzilla : 11223
Details : Change "dropped message" CERRORs to D_NETERROR so they are
logged instead of creating "console chatter" when a lustre
timeout races with normal RPC completion.
Severity : minor
Details : lnet_clear_peer_table can wait forever if user forgets to
clear a lazy portal.
Severity : minor
Details : libcfs_id2str should check pid against LNET_PID_ANY.
Severity : major
Bugzilla : 10916
Description: added LNET self test
Details : landing b_self_test
Severity : minor
Frequency : rare
Bugzilla : 12227
Description: cfs_duration_{u,n}sec() wrongly calculate nanosecond part of
struct timeval.
Details : do_div() macro is used incorrectly.
2007-04-23 Cluster File Systems, Inc. <info@clusterfs.com>
Severity : normal
Bugzilla : 11680
Description: make panic on lbug configurable
Severity : major
Bugzilla : 12316
Description: Add OFED1.2 support to o2iblnd
Details : o2iblnd depends on OFED's modules, if out-tree OFED's modules
are installed (other than kernel's in-tree infiniband), there
could be some problem while insmod o2iblnd (mismatch CRC of
ib_* symbols).
If extra Module.symvers is supported in kernel (i.e, 2.6.17),
this link provides solution:
https://bugs.openfabrics.org/show_bug.cgi?id=355
if extra Module.symvers is not supported in kernel, we will
have to run the script in bug 12316 to update
$LINUX/module.symvers before building o2iblnd.
More details about this are in bug 12316.
------------------------------------------------------------------------------
2007-04-01 Cluster File Systems, Inc. <info@clusterfs.com>
* version 1.4.10 / 1.6.0
* Support for networks:
socklnd - kernels up to 2.6.16,
qswlnd - Qsnet kernel modules 5.20 and later,
openiblnd - IbGold 1.8.2,
o2iblnd - OFED 1.1,
viblnd - Voltaire ibhost 3.4.5 and later,
ciblnd - Topspin 3.2.0,
iiblnd - Infiniserv 3.3 + PathBits patch,
gmlnd - GM 2.1.22 and later,
mxlnd - MX 1.2.1 or later,
ptllnd - Portals 3.3 / UNICOS/lc 1.5.x, 2.0.x
Severity : minor
Frequency : rare
Description: Ptllnd didn't init kptllnd_data.kptl_idle_txs before it could be
possibly accessed in kptllnd_shutdown. Ptllnd should init
kptllnd_data.kptl_ptlid2str_lock before calling kptllnd_ptlid2str.
Severity : normal
Frequency : rare
Description: gmlnd ignored some transmit errors when finalizing lnet messages.
Severity : minor
Frequency : rare
Description: ptllnd logs a piece of incorrect debug info in kptllnd_peer_handle_hello.
Severity : minor
Frequency : rare
Description: the_lnet.ln_finalizing was not set when the current thread is
about to complete messages. It only affects multi-threaded
user space LNet.
Severity : normal
Frequency : rare
Bugzilla : 11472
Description: Changed the default kqswlnd ntxmsg=512
Severity : major
Frequency : rare
Bugzilla : 12458
Description: Assertion failure in kernel ptllnd caused by posting passive
bulk buffers before connection establishment complete.
Severity : major
Frequency : rare
Bugzilla : 12445
Description: A race in kernel ptllnd between deleting a peer and posting
new communications for it could hang communications -
manifesting as "Unexpectedly long timeout" messages.
Severity : major
Frequency : rare
Bugzilla : 12432
Description: Kernel ptllnd lock ordering issue could hang a node.
Severity : major
Frequency : rare
Bugzilla : 12016
Description: node crash on socket teardown race
Severity : minor
Frequency : 'lctl peer_list' issued on a mx net
Bugzilla : 12237
Description: Enable lctl's peer_list for MXLND
Severity : major
Frequency : after Ptllnd timeouts and portals congestion
Bugzilla : 11659
Description: Credit overflows
Details : This was a bug in ptllnd connection establishment. The fix
implements better peer stamps to disambiguate connection
establishment and ensure both peers enter the credit flow
state machine consistently.
Severity : major
Frequency : rare
Bugzilla : 11394
Description: kptllnd didn't propagate some network errors up to LNET
Details : This bug was spotted while investigating 11394. The fix
ensures network errors on sends and bulk transfers are
propagated to LNET/lustre correctly.
Severity : enhancement
Bugzilla : 10316
Description: Fixed console chatter in case of -ETIMEDOUT.
Severity : enhancement
Bugzilla : 11684
Description: Added D_NETTRACE for recording network packet history
(initially only for ptllnd). Also a separate userspace
ptllnd facility to gather history which should really be
covered by D_NETTRACE too, if only CDEBUG recorded history in
userspace.
Severity : major
Frequency : rare
Bugzilla : 11616
Description: o2iblnd handle early RDMA_CM_EVENT_DISCONNECTED.
Details : If the fabric is lossy, an RDMA_CM_EVENT_DISCONNECTED
callback can occur before a connection has actually been
established. This caused an assertion failure previously.
Severity : enhancement
Bugzilla : 11094
Description: Multiple instances for o2iblnd
Details : Allow multiple instances of o2iblnd to enable networking over
multiple HCAs and routing between them.
Severity : major
Bugzilla : 11201
Description: lnet deadlock in router_checker
Details : turned ksnd_connd_lock, ksnd_reaper_lock, and ksock_net_t:ksnd_lock
into BH locks to eliminate potential deadlock caused by
ksocknal_data_ready() preempting code holding these locks.
Severity : major
Bugzilla : 11126
Description: Millions of failed socklnd connection attempts cause a very slow FS
Details : added a new route flag ksnr_scheduled to distinguish from
ksnr_connecting, so that a peer connection request is only turned
down for race concerns when an active connection to the same peer
is under progress (instead of just being scheduled).
------------------------------------------------------------------------------
2007-02-09 Cluster File Systems, Inc. <info@clusterfs.com>
* version 1.4.9
* Support for networks:
socklnd - kernels up to 2.6.16
qswlnd - Qsnet kernel modules 5.20 and later
openiblnd - IbGold 1.8.2
o2iblnd - OFED 1.1
viblnd - Voltaire ibhost 3.4.5 and later
ciblnd - Topspin 3.2.0
iiblnd - Infiniserv 3.3 + PathBits patch
gmlnd - GM 2.1.22 and later
mxlnd - MX 1.2.1 or later
ptllnd - Portals 3.3 / UNICOS/lc 1.5.x, 2.0.x
* bug fixes
Severity : major on XT3
Bugzilla : none
Description: libcfs overwrites /proc/sys/portals
Details : libcfs created a symlink from /proc/sys/portals to
/proc/sys/lnet for backwards compatibility. This is no
longer required and makes the Cray portals /proc variables
inaccessible.
Severity : minor
Bugzilla : 11312
Description: OFED FMR API change
Details : This changes parameter usage to reflect a change in
ib_fmr_pool_map_phys() between OFED 1.0 and OFED 1.1. Note
that FMR support is only used in experimental versions of the
o2iblnd - this change does not affect standard usage at all.
Severity : enhancement
Bugzilla : 11245
Description: new ko2iblnd module parameter: ib_mtu
Details : the default IB MTU of 2048 performs badly on 23108 Tavor
HCAs. You can avoid this problem by setting the MTU to 1024
using this module parameter.
Severity : enhancement
Bugzilla : 11118/11620
Description: ptllnd small request message buffer alignment fix
Details : Set the PTL_MD_LOCAL_ALIGN8 option on small message receives.
Round up small message size on sends in case this option
is not supported. 11620 was a defect in the initial
implementation which effectively asserted all peers had to be
running the correct protocol version which was fixed by always
NAK-ing such requests and handling any misalignments they
introduce.
Severity : minor
Frequency : rarely
Description: When kib(nal|lnd)_del_peer() is called upon a peer whose
ibp_tx_queue is not empty, kib(nal|lnd)_destroy_peer()'s
'LASSERT(list_empty(&peer->ibp_tx_queue))' will fail.
Severity : enhancement
Bugzilla : 11250
Description: Patchless ZC(zero copy) socklnd
Details : New protocol for socklnd, socklnd can support zero copy without
kernel patch, it's compatible with old socklnd. Checksum is
moved from tunables to modparams.
Severity : minor
Frequency : rarely
Description: When ksocknal_del_peer() is called upon a peer whose
ksnp_tx_queue is not empty, ksocknal_destroy_peer()'s
'LASSERT(list_empty(&peer->ksnp_tx_queue))' will fail.
Severity : normal
Frequency : when ptlrpc is under heavy use and runs out of request buffer
Bugzilla : 11318
Description: In lnet_match_blocked_msg(), md can be used without holding a
ref on it.
Severity : minor
Frequency : very rarely
Bugzilla : 10727
Description: If ksocknal_lib_setup_sock() fails, a ref on peer is lost.
If connd connects a route which has been closed by
ksocknal_shutdown(), ksocknal_create_routes() may create new
routes which hold references on the peer, causing shutdown
process to wait for peer to disappear forever.
Severity : enhancement
Bugzilla : 11234
Description: Dump XT3 portals traces on kptllnd timeout
Details : Set the kptllnd module parameter "ptltrace_on_timeout=1" to
dump Cray portals debug traces to a file. The kptllnd module
parameter "ptltrace_basename", default "/tmp/lnet-ptltrace",
is the basename of the dump file.
Severity : major
Frequency : infrequent
Bugzilla : 11308
Description: kernel ptllnd fix bug in connection re-establishment
Details : Kernel ptllnd could produce protocol errors e.g. illegal
matchbits and/or violate the credit flow protocol when trying
to re-establish a connection with a peer after an error or
timeout.
Severity : enhancement
Bugzilla : 10316
Description: Allow /proc/sys/lnet/debug to be set symbolically
Details : Allow debug and subsystem debug values to be read/set by name
in addition to numerically, for ease of use.
Severity : normal
Frequency : only in configurations with LNET routers
Bugzilla : 10316
Description: routes automatically marked down and recovered
Details : In configurations with LNET routers if a router fails routers
now actively try to recover routes that are down, unless they
are marked down by an administrator.
------------------------------------------------------------------------------
2006-12-09 Cluster File Systems, Inc. <info@clusterfs.com>
Severity : critical
Frequency : very rarely, in configurations with LNET routers and TCP
Bugzilla : 10889
Description: incorrect data written to files on OSTs
Details : In certain high-load conditions incorrect data may be written
to files on the OST when using TCP networks.
------------------------------------------------------------------------------
2006-07-31 Cluster File Systems, Inc. <info@clusterfs.com>
* version 1.4.7
- rework CDEBUG messages rate-limiting mechanism b=10375
- add per-socket tunables for socklnd if the kernel is patched b=10327
------------------------------------------------------------------------------
2006-02-15 Cluster File Systems, Inc. <info@clusterfs.com>
* version 1.4.6
- fix use of portals/lnet pid to avoid dropping RPCs b=10074
- iiblnd wasn't mapping all memory, resulting in comms errors b=9776
- quiet LNET startup LNI message for liblustre b=10128
- Better console error messages if 'ip2nets' can't match an IP address
- Fixed overflow/use-before-set bugs in linux-time.h
- Fixed ptllnd bug that wasn't initialising rx descriptors completely
- LNET teardown failed an assertion about the route table being empty
- Fixed a crash in LNetEQPoll(<invalid handle>)
- Future protocol compatibility work (b_rls146_lnetprotovrsn)
- improve debug message for liblustre/Catamount nodes (b=10116)
2005-10-10 Cluster File Systems, Inc. <info@clusterfs.com>
* Configuration change for the XT3
The PTLLND is now used to run Lustre over Portals on the XT3.
The configure option(s) --with-cray-portals are no longer
used. Rather --with-portals=<path-to-portals-includes> is
used to enable building on the XT3. In addition to enable
XT3 specific features the option --enable-cray-xt3 must be
used.
2005-10-10 Cluster File Systems, Inc. <info@clusterfs.com>
* Portals has been removed, replaced by LNET.
LNET is new networking infrastructure for Lustre, it includes a
reorganized network configuration mode (see the user
documentation for full details) as well as support for routing
between different network fabrics. Lustre Networking Devices
(LNDS) for the supported network fabrics have also been created
for this new infrastructure.
2005-08-08 Cluster File Systems, Inc. <info@clusterfs.com>
* version 1.4.4
* bug fixes
Severity : major
Frequency : rare (large Voltaire clusters only)
Bugzilla : 6993
Description: the default number of reserved transmit descriptors was too low
for some large clusters
Details : As a workaround, the number was increased. A proper fix includes
a run-time tunable.
2005-06-02 Cluster File Systems, Inc. <info@clusterfs.com>
* version 1.4.3
* bug fixes
Severity : major
Frequency : occasional (large-scale events, cluster reboot, network failure)
Bugzilla : 6411
Description: too many error messages on console obscure actual problem and
can slow down/panic server, or cause recovery to fail repeatedly
Details : enable rate-limiting of console error messages, and some messages
that were console errors now only go to the kernel log
Severity : enhancement
Bugzilla : 1693
Description: add /proc/sys/portals/catastrophe entry which will report if
that node has previously LBUGged
2005-04-06 Cluster File Systems, Inc. <info@clusterfs.com>
* bugs
- update gmnal to use PTL_MTU, fix module refcounting (b=5786)
2005-04-04 Cluster File Systems, Inc. <info@clusterfs.com>
* bugs
- handle error return code in kranal_check_fma_rx() (5915,6054)
2005-02-04 Cluster File Systems, Inc. <info@clusterfs.com>
* miscellania
- update vibnal (Voltaire IB NAL)
- update gmnal (Myrinet NAL), gmnalid
2005-02-04 Eric Barton <eeb@bartonsoftware.com>
* Landed portals:b_port_step as follows...
- removed CFS_DECL_SPIN*
just use 'spinlock_t' and initialise with spin_lock_init()
- removed CFS_DECL_MUTEX*
just use 'struct semaphore' and initialise with init_mutex()
- removed CFS_DECL_RWSEM*
just use 'struct rw_semaphore' and initialise with init_rwsem()
- renamed cfs_sleep_chan -> cfs_waitq
cfs_sleep_link -> cfs_waitlink
- fixed race in linux version of arch-independent socknal
(the ENOMEM/EAGAIN decision).
- Didn't fix problems in Darwin version of arch-independent socknal
(resetting socket callbacks, eager ack hack, ENOMEM/EAGAIN decision)
- removed libcfs types from non-socknal header files (only some types
in the header files had been changed; the .c files hadn't been
updated at all).
