[Lustre-discuss] MDT and MGS

[Jeff Darcy]

Brian J. Murrell wrote:
> But you will notice that in the specific section 4.2.2.6 "Running
> Multiple Lustre Filesystems" they do demonstrate setting up a separate
> MGS:
>   
Actually they demonstrate setting up a separate MDT to use an existing 
MGS.  There's even a note that says "specify --mdt --mgs on one, and 
--mdt --mgsnode=/<mgsnodenid>/ on the others" which would still run into 
the same problem should one ever need to reformat that first MGS/MDT.  
That won't direct somebody who has "even a notion of wanting more than 
one filesystem" toward creating a separate MGS.  If it's something you 
guys feel should be recommended for all or nearly all cases, it needs to 
be presented that way in even the early examples.
>> Has 
>> any consideration been given to ways of storing the MGS information 
>> within a file on an existing filesystem instead of requiring on a 
>> separate block device?
>>     
>
> How would that be any better than the co-located MGS/MDT situation where
> you want to reformat the filesystem that has the configuration
> information stored on it in a file?
>   
That's why I said an *existing* filesystem - i.e. one existing before 
and therefore separate from any MDTs you create.  Sure, if you reformat 
the filesystem containing the MGS data you'll still have to do a 
(trivial) save and restore, but that will always be the case no matter 
where the MGS data goes.  At least customers wouldn't find themselves 
facing the problem that started the thread, where reformatting the MDT 
will rather unexpectedly mean reformatting their MGS as well.

> But you still have the problem of formatting that particular filesystem.
> I suppose you could umount the loopback device, copy the file to a
> different filesystem and remount the loopback device.  That just seems
> cumbersome.
>   
Yes, it is, but no more so than the MGS-on-private-storage case 
currently.  More importantly, it avoids the cumbersome requirement to 
devote a whole separate block device to this role.  Customers' notions 
of which burden to avoid might not match ours, and I've found that many 
customers really loathe allocating tiny LUNs for special purposes 
(gateway LUNs on EMC/Clariion devices are another example).  This is 
something a customer could do today without code changes to work around 
that particular burden.