Status History

System Name Status Details ETR Timestamp User ID
High Performance Primary Storage Available
-- 2013-10-29 09:09:39 mbaynton
High Performance Primary Storage Down

MSI home directory storage is down for a scheduled software upgrade.

2013-12-04 06:30:00 2013-12-04 05:26:15 vestrum
High Performance Primary Storage Available

Functioning Normally.

-- 2013-12-04 07:49:48 vestrum
High Performance Primary Storage Intermittent

Some volumes responding very slowly, affecting about 10% of users. Troubleshooting with the storage vendor.

-- 2013-12-09 13:08:48 vestrum
High Performance Primary Storage Available

Functioning Normally.

-- 2013-12-09 13:58:03 vestrum
High Performance Primary Storage Intermittent

Several volumes are slow or unavailable, affecting a minority of users. MSI administrators are working to restore.

-- 2014-04-07 11:02:33 mbaynton
High Performance Primary Storage Available
-- 2014-04-07 16:12:07 mbaynton
High Performance Primary Storage Intermittent

File system is currently slowed due to a hardware failure

-- 2014-05-01 10:19:07 geron018
High Performance Primary Storage Available
-- 2014-05-01 12:59:07 badger
High Performance Primary Storage Available

Quotas are being reinitialized. groupquota may return incorrect results and disk usage warning emails are suspended during this time.

-- 2014-07-10 10:11:45 mbaynton
High Performance Primary Storage Available
-- 2014-07-11 08:36:25 mbaynton
High Performance Primary Storage Intermittent

Home directories of approx. 1/15th of users are offline due to emergency maintenance

2014-07-19 17:00:00 2014-07-19 14:17:17 mbaynton
High Performance Primary Storage Available

All home directories available after emergency maintenance 2014-07-19

-- 2014-07-19 15:04:39 vestrum
High Performance Primary Storage Available
-- 2014-07-19 15:24:24 mbaynton
High Performance Primary Storage Intermittent

Panasas volume /groups/13 is offline for file system repairs

2014-10-26 06:00:00 2014-10-25 23:02:37 vestrum
High Performance Primary Storage Available
-- 2014-10-26 07:34:59 vestrum
High Performance Primary Storage Down

Upgrading and expanding Panasas storage. I/O will hang during the upgrade.

2014-11-05 07:00:00 2014-11-05 05:03:31 vestrum
High Performance Primary Storage Available
-- 2014-11-05 08:11:51 vestrum
High Performance Primary Storage Intermittent

Intermittant issue resulting in 2-3 minute hangs on some volumes. MSI is actively working with the storage vendor on this issue

-- 2014-11-10 11:20:18 vestrum
High Performance Primary Storage Available
-- 2014-11-11 13:44:18 vestrum
High Performance Primary Storage Down

Unscheduled downtime

-- 2014-11-12 08:40:37 rathx082
High Performance Primary Storage Available
-- 2014-11-12 09:06:04 vestrum
High Performance Primary Storage Down

Troubleshooting with the vendor

-- 2015-04-24 08:08:00 mbaynton
High Performance Primary Storage Available
-- 2015-04-24 10:02:25 mbaynton
High Performance Primary Storage Down

Applying OS and related updates per regular maintenance

2015-05-05 19:00:00 2015-05-06 05:17:18 nhospoda
High Performance Primary Storage Available
-- 2015-05-06 15:04:15 aring
High Performance Primary Storage Intermittent

maintenance day - you may experience system interruptions

-- 2015-06-02 16:37:16 tekla010
High Performance Primary Storage Available
-- 2015-06-03 14:08:35 santo136
High Performance Primary Storage Intermittent

tomorrow is maintenance day - you may experience system interruptions

-- 2015-06-30 17:07:53 jubar001
High Performance Primary Storage Intermittent

July 1st is maintenance day - you may experience system interruptions

-- 2015-06-30 17:20:18 jubar001
High Performance Primary Storage Available
-- 2015-07-01 15:47:56 ofuon001
High Performance Primary Storage Available

Scheduled maintenance is Oct 7.

2015-10-07 16:00:00 2015-09-29 12:18:15 aring
High Performance Primary Storage Intermittent

Wed, Oct 7 is maintenance day. Except intermittent issues.

-- 2015-10-06 16:01:54 jubar001
High Performance Primary Storage Available

Normal Function

-- 2015-10-07 16:01:54 aring
High Performance Primary Storage Intermittent

intermittent: file read and write is slow

-- 2015-10-21 14:29:29 ensig011
High Performance Primary Storage Intermittent

intermittent: file read and write is slow, may have problems logging in

-- 2015-11-04 09:40:34 herxx430
High Performance Primary Storage Intermittent

intermittent: File read and write slow. May have problems logging in. Some software licenses unavailable.

-- 2015-11-04 19:59:18 rowe
High Performance Primary Storage Intermittent

MSI Systems will be down for maintenance on 11/11/2015

-- 2015-11-10 15:14:59 ofuon001
High Performance Primary Storage Available
-- 2015-11-20 09:06:44 mbaynton
High Performance Primary Storage Intermittent

global /scratch undergoing maintenance

2015-12-02 14:00:00 2015-12-02 09:29:35 nhospoda
High Performance Primary Storage Available

maintenance is complete

-- 2015-12-02 09:51:39 nhospoda
High Performance Primary Storage Intermittent

Troubleshooting performance issues with vendor. Access to home directories may periodically hang.

-- 2015-12-14 11:44:35 mbaynton
High Performance Primary Storage Available
-- 2015-12-15 13:50:28 nhospoda
High Performance Primary Storage Intermittent

Systems will be unavalible during schedualed maintenace on January 6.

2016-01-06 16:00:00 2016-01-04 15:19:55 aring
High Performance Primary Storage Available
-- 2016-01-06 16:09:06 aring
High Performance Primary Storage Intermittent

Filesystem is running very slowly.

-- 2016-01-26 12:26:40 jubar001
High Performance Primary Storage Available
-- 2016-02-02 11:12:28 jubar001
High Performance Primary Storage Available

Filesystem operation restored to normal.

-- 2016-02-02 11:15:40 jubar001
High Performance Primary Storage Intermittent

Maintenance Day Wednesday, February 3

-- 2016-02-02 13:18:45 ofuon001
High Performance Primary Storage Intermittent

Maintenance Day Wednesday, February 3

-- 2016-02-02 13:26:53 mbaynton
High Performance Primary Storage Available
-- 2016-02-04 14:44:51 nhospoda
High Performance Primary Storage Intermittent

Maintenance Day is Wednesday, March 2

2016-03-02 16:00:00 2016-03-01 16:56:43 rowe
High Performance Primary Storage Intermittent

Maintenance is extended through end of day. Users may experience occasional lags in performace

2016-03-03 16:00:00 2016-03-03 09:24:08 chaux034
High Performance Primary Storage Available

High Performance Storage should now be operating normally.

-- 2016-03-03 13:47:57 nhospoda
High Performance Primary Storage Intermittent

High Performance Storage is undergoing volume reconstrution which may cause some slow performance

2016-03-27 00:00:00 2016-03-25 16:16:59 root
High Performance Primary Storage Intermittent

High Performance Storage is undergoing volume reconstrution which may cause some slow performance

2016-03-27 00:00:00 2016-03-25 16:17:19 nhospoda
High Performance Primary Storage Available

High Performance Storage should now be operating normally.

-- 2016-03-26 09:52:54 nhospoda
High Performance Primary Storage Intermittent

Experiencing slow performance on some volumes

-- 2016-04-04 08:33:11 mbaynton
High Performance Primary Storage Intermittent

Panasas storage maintenance will affect all users and prevent access to all systems from 04:00 - 16:00 CST.

-- 2016-04-05 12:41:17 jubar001
High Performance Primary Storage Available
-- 2016-04-06 17:08:53 nhospoda
High Performance Primary Storage Available
-- 2016-04-11 16:37:13 wendt155
High Performance Primary Storage Available
-- 2016-04-22 14:18:37 wendt155
High Performance Primary Storage Intermittent

Maintenance Day Wednesday, May 4

-- 2016-05-04 09:47:30 chaux034
High Performance Primary Storage Available
-- 2016-05-04 16:51:02 nhospoda
High Performance Primary Storage Intermittent

Maintenance Day is Wednesday, June 1

-- 2016-06-01 11:01:39 chaux034
High Performance Primary Storage Available
-- 2016-06-01 17:11:07 nhospoda
High Performance Primary Storage Available
-- 2016-07-21 14:38:02 dowel015
High Performance Primary Storage Intermittent

Blade reconstuction in process resulting in mild slowness throughout the morning

-- 2016-08-22 09:53:28 leibfrie
High Performance Primary Storage Available
-- 2016-08-22 13:18:35 leibfrie
High Performance Primary Storage Down

maintenance day updates

-- 2016-09-07 11:12:11 nhospoda
High Performance Primary Storage Intermittent

Some Panasas home directories are still undergoing maintanence and are unavailable

-- 2016-09-07 16:31:37 root
High Performance Primary Storage Intermittent

Some Panasas home directories are still undergoing maintanence and are unavailable

-- 2016-09-07 16:20:37 root
High Performance Primary Storage Available
-- 2016-09-08 08:02:26 leibfrie
High Performance Primary Storage Available
-- 2016-09-13 08:23:49 dowel015
High Performance Primary Storage Intermittent

Panasas is having difficulties

-- 2016-09-20 11:26:24 klp
High Performance Primary Storage Intermittent

Primary storage incident is affecting some users on the system

-- 2016-09-22 08:34:54 nscott
High Performance Primary Storage Available
-- 2016-09-24 07:04:42 nhospoda
High Performance Primary Storage Available
-- 2016-09-30 08:35:03 dowel015
High Performance Primary Storage Down

monthly maintenance

-- 2016-10-05 07:04:26 nhospoda
High Performance Primary Storage Available

maintenance complete

-- 2016-10-05 10:12:47 root
High Performance Primary Storage Available

maintenance complete

-- 2016-10-05 10:13:02 nhospoda
High Performance Primary Storage Intermittent

Some volumes are intermittent causing delays.

-- 2016-10-11 15:03:50 nscott
High Performance Primary Storage Intermittent

Some volumes are intermittent, causing delays.

-- 2016-10-11 15:04:34 nscott
High Performance Primary Storage Intermittent

Some volumes are intermittent, causing delays. Currently working with the vendor, no ETA yet.

-- 2016-10-11 15:05:24 nscott
High Performance Primary Storage Available

Issues have been resolved. System is stable and completely back online.

-- 2016-10-12 15:58:58 nscott
High Performance Primary Storage Intermittent

Failed hardware is slowing down Panasas while it is recovering and rebuilding from the loss. A replacement part is inbound.

-- 2016-11-01 11:08:06 leibfrie
High Performance Primary Storage Down

A hardware failure has caused several volumes to become unavilable

-- 2016-11-02 17:24:41 root
High Performance Primary Storage Intermittent

A hardware failure has caused several volumes to become unavilable

-- 2016-11-03 16:07:23 nscott
High Performance Primary Storage Available

All filesystems are back online

-- 2016-11-04 09:44:03 nhospoda
High Performance Primary Storage Intermittent

Maintenance Day Wednesday, December 7

-- 2016-12-07 08:55:03 nscott
High Performance Primary Storage Available
-- 2016-12-07 15:33:43 nhospoda
High Performance Primary Storage Intermittent

Maintenance day

-- 2017-01-04 09:06:45 mbaynton
High Performance Primary Storage Intermittent

Maintenance Day Wednesday, February 3

-- 2017-01-05 15:42:23 nscott
High Performance Primary Storage Available
-- 2017-01-05 15:44:47 nscott
High Performance Primary Storage Intermittent

Maintenance Day

-- 2017-03-01 09:20:31 nscott
High Performance Primary Storage Available
-- 2017-03-02 10:22:39 nscott
High Performance Primary Storage Intermittent

The Globus endpoint to High Performance Storage is currently experiencing intermittent timeouts.

-- 2017-03-16 12:16:06 dunn0404
High Performance Primary Storage Intermittent

Maintenance Day Wednesday, April 5

-- 2017-04-05 08:44:00 nscott
High Performance Primary Storage Available
-- 2017-04-06 08:53:43 nscott
High Performance Primary Storage Intermittent

Maintenance Day

-- 2017-06-07 06:50:41 mattmix