On Jan 29, 2011, at 6:04 AM, carlopmart carlopmart@gmail.com wrote:

On 01/28/2011 03:21 PM, Ross Walker wrote:

...

On Jan 28, 2011, at 4:55 AM, carlopmartcarlopmart@gmail.com wrote:

...

Hi all,

I need to install a virtual machine acting as a virtual storage server under CentOS 5.x (using kvm, xen, virtualbox or vmware). This virtual storage machine needs to server storage to another ESXi server and at the same time to the host where is installed.

This is due to the limitations of hardware I have available. Both hosts needs to server several machines.

It is very important that the virtual machine consumes the least resources possible (host has 5GB RAM and i need to run three virtual machines minimum, including this storage server as a virtual machine).

What can be better solution: CentOS, NexentaStor, openfiler ...??

For such a small setup, I recommend installing ESXi on both machines and setting up a storage server on the ESXi box with all the storage.

Use NFS for your storage server. Disable ESXi memory ballooning/over commit for your storage VM otherwise you'll have memory contention between storage producer and storage consumers.

Your choice of OS depends on your experience level and needs. If your comfortable with Redhat Linux use CentOS minimal install, otherwise use OpenFiler. If data integrity is more important then performance use Nexentastor (if performance is more important then consistency disable ZIL, ZFS guarantees file system integrity, ZIL guarantees data consistency).

-Ross

Thanks Ross. I had been thought about this solution. But, there is a problem: I need to run two more VMs on that server and only has 5GB of RAM. AFAIK, NexentaStor requires a minimum of 4GB of RAM.

But If I use nfs services to share disks: can I limit memory used by nfs process in some manner??

What OS are the VMs?

If they are windows, then I'd just use Microsoft SBS and run all services off one box then instead of multiple VMs.

If they are Linux, think about using a container based solution like OpenVZ.

5GB is only enough memory to run 1 or 2 VMs.

-Ross

On 01/29/11 11:42 AM, carlopmart wrote:

All OS will be UNix based: linux, BSD or Solaris ...

Solaris is by design quite memory intensive, since on modern servers, memory is cheap.

ZFS in particular is designed to use large amounts of memory to optimize storage performance.

----- Original Message ----- | ----- Original Message ----- | | On 1/28/2011 3:55 AM, carlopmart wrote: | | > Hi all, | | > | | > I need to install a virtual machine acting as a virtual storage | | > server under | | > CentOS 5.x (using kvm, xen, virtualbox or vmware). This virtual | | > storage machine | | > needs to server storage to another ESXi server and at the same | | > time | | > to the host | | > where is installed. | | > | | > This is due to the limitations of hardware I have available. | | > Both | | > hosts needs to | | > server several machines. | | > | | > It is very important that the virtual machine consumes the | | > least | | > resources | | > possible (host has 5GB RAM and i need to run three virtual | | > machines | | > minimum, | | > including this storage server as a virtual machine). | | | | What's the point of adding an extra virtual layer compared to an nfs | | or | | iscsi share from the host (nfs if it is shared, iscsi if it is the | | VM | | image store)? This seems like it would be more efficient if you run | | exsi on the hardware with centos and the others as guests anyway. | | | | -- | | Les Mikesell | | lesmikesell@gmail.com | | _______________________________________________ | | CentOS mailing list | | CentOS@centos.org | | http://lists.centos.org/mailman/listinfo/centos | | There are some advantages that I can see in that if your hardware dies | you can migrate the entire host and disks over to another VMWare | hosts. | | If your NFS host is not H/A a loss of the host would take down the | virtual machines too. Additionally, virtualization offers the ability | to migrate the VM and disk to newer hardware somewhat transparently | allowing you to take advantage of the latest/greatest/buggy tech. | | Just my 2c ;) | | -- | James A. Peltier | IT Services - Research Computing Group | Simon Fraser University - Burnaby Campus | Phone : 778-782-6573 | Fax : 778-782-3045 | E-Mail : jpeltier@sfu.ca | Website : http://www.sfu.ca/itservices | http://blogs.sfu.ca/people/jpeltier | | | _______________________________________________ | CentOS mailing list | CentOS@centos.org | http://lists.centos.org/mailman/listinfo/centos

I should add though that you will likely see a rather large performance penalty on the I/O side

On 1/29/11 5:05 AM, carlopmart wrote:

...

|> It is very important that the virtual machine consumes the least |> resources |> possible (host has 5GB RAM and i need to run three virtual machines |> minimum, |> including this storage server as a virtual machine). | | What's the point of adding an extra virtual layer compared to an nfs | or | iscsi share from the host (nfs if it is shared, iscsi if it is the VM | image store)? This seems like it would be more efficient if you run | exsi on the hardware with centos and the others as guests anyway. |

There are some advantages that I can see in that if your hardware dies you can migrate the entire host and disks over to another VMWare hosts.

If your NFS host is not H/A a loss of the host would take down the virtual machines too. Additionally, virtualization offers the ability to migrate the VM and disk to newer hardware somewhat transparently allowing you to take advantage of the latest/greatest/buggy tech.

Just my 2c ;)

Correct.

But I don't see how any of those things apply here. If the host fails your vm's are going to fail in any case, and there's not much magic involved in exporting an NFS share even if you need to move it. Iscsi targets are slightly more complicated because it's not included in the base Centos install but you can find howto's to set it up. When your resources are limited it looks like a big waste to add an unnecessary virtual layer to storage. I've done it the other way around, though, with NFS exports from the host being mounted by the guest VM's.

----- Original Message ----- | On 1/29/11 5:05 AM, carlopmart wrote: | >> | >> |> It is very important that the virtual machine consumes the | >> |> least | >> |> resources | >> |> possible (host has 5GB RAM and i need to run three virtual | >> |> machines | >> |> minimum, | >> |> including this storage server as a virtual machine). | >> | | >> | What's the point of adding an extra virtual layer compared to an | >> | nfs | >> | or | >> | iscsi share from the host (nfs if it is shared, iscsi if it is | >> | the VM | >> | image store)? This seems like it would be more efficient if you | >> | run | >> | exsi on the hardware with centos and the others as guests anyway. | >> | | >> | >> There are some advantages that I can see in that if your hardware | >> dies you can migrate the entire host and disks over to another | >> VMWare hosts. | >> | >> If your NFS host is not H/A a loss of the host would take down the | >> virtual machines too. Additionally, virtualization offers the | >> ability to migrate the VM and disk to newer hardware somewhat | >> transparently allowing you to take advantage of the | >> latest/greatest/buggy tech. | >> | >> Just my 2c ;) | >> | > | > Correct. | | But I don't see how any of those things apply here. If the host fails | your vm's | are going to fail in any case, and there's not much magic involved in | exporting | an NFS share even if you need to move it. Iscsi targets are slightly | more | complicated because it's not included in the base Centos install but | you can | find howto's to set it up. When your resources are limited it looks | like a big | waste to add an unnecessary virtual layer to storage. I've done it the | other | way around, though, with NFS exports from the host being mounted by | the guest VM's. | | -- | Les Mikesell | lesmikesell@gmail.com

I made no claims that it solved anything. I merely noted why someone might want to virtualize in place of NFS. Personally, I don't think that the OP really knows what they want, or they want the best of all worlds without compromise. I don't see how it is possible to provide what is being asked for. Really I think a minimum of two ideally a third server providing iSCSI or NFS is needed for the solution to work. That third machine should have all of the possible host level redundancy possible to keep it running. If H/A is required at least two machines are required.

On 01/31/2011 04:05 AM, James A. Peltier wrote:

----- Original Message -----> |>> |> |> Correct. | | But I don't see how any of those things apply here. If the host fails | your vm's | are going to fail in any case, and there's not much magic involved in | exporting | an NFS share even if you need to move it. Iscsi targets are slightly | more | complicated because it's not included in the base Centos install but | you can | find howto's to set it up. When your resources are limited it looks | like a big | waste to add an unnecessary virtual layer to storage. I've done it the | other | way around, though, with NFS exports from the host being mounted by | the guest VM's. | | -- | Les Mikesell | lesmikesell@gmail.com

I made no claims that it solved anything. I merely noted why someone might want to virtualize in place of NFS. Personally, I don't think that the OP really knows what they want, or they want the best of all worlds without compromise. I don't see how it is possible to provide what is being asked for. Really I think a minimum of two ideally a third server providing iSCSI or NFS is needed for the solution to work. That third machine should have all of the possible host level redundancy possible to keep it running. If H/A is required at least two machines are required.

Ok I will try to explain with more details. First, this installation it is for my home personal use, It isn't for a production environment 24x7 or similar.

I have two physical hosts with this configuration:

HostA:

- HP ML150 - 5GB RAM - 3TB for storage with HP smartArray E200i - Intel Xeon QuadCore.

HostB:

- HP ML115 G5 - 8GB RAM - 160GB for storage - AMD QuadCore

Ok, lets go. I need (or I will like to do) to setup several virtual machines to accomplish different tasks (remeber, It is for personal use, like a lab environment):

- 1 virtual machine using as a DNS server and Kerberos authentication (CentOS or RedHat) - 2 virtual machines with RHCS installed providing several services: smtp server (only smtp), mirror updates, squid and cifs server. (with CentOS5) - 1 virtual machine with Windows 7 as a workstation. - 1 virtual machine with Windows 2008 R2 server. - 2 virtual machines with RHCS installed with OSSEC. Snort. Snortby and Splunk server (with CentOS5 too) - 2 virtual machines with OpenBSD firewalls with CARP and load balancing. - 1 virtual machine as a DMZ Server.

My idea is to install DNS server (with kerberos auth) and 2 virtual virtual machines with RHCS and common services linke smtp, squid, etc onto HP ML150. And the others virtual machines running on HP ML115 server.

Where is the problem?? Problem is the storage. All storage resides on the HP ML150 server. For that reason I need to install a server as a virtual storage to run most of the virtual machines running on the server HP ML115 with the exception of firewalls and the DMZ server that resides on the HP ML115's local disk.

For backups I have an external usb disk with 1TB.

Thanks.

On Jan 31, 2011, at 8:58 AM, Les Mikesell lesmikesell@gmail.com wrote:

On 1/31/11 3:20 AM, carlopmart wrote:

...

On 01/31/2011 04:05 AM, James A. Peltier wrote:

...

----- Original Message -----> |>> |> |> Correct. | | But I don't see how any of those things apply here. If the host fails | your vm's | are going to fail in any case, and there's not much magic involved in | exporting | an NFS share even if you need to move it. Iscsi targets are slightly | more | complicated because it's not included in the base Centos install but | you can | find howto's to set it up. When your resources are limited it looks | like a big | waste to add an unnecessary virtual layer to storage. I've done it the | other | way around, though, with NFS exports from the host being mounted by | the guest VM's. | | -- | Les Mikesell | lesmikesell@gmail.com

I made no claims that it solved anything. I merely noted why someone might want to virtualize in place of NFS. Personally, I don't think that the OP really knows what they want, or they want the best of all worlds without compromise. I don't see how it is possible to provide what is being asked for. Really I think a minimum of two ideally a third server providing iSCSI or NFS is needed for the solution to work. That third machine should have all of the possible host level redundancy possible to keep it running. If H/A is required at least two machines are required.

Ok I will try to explain with more details. First, this installation it is for my home personal use, It isn't for a production environment 24x7 or similar.

I have two physical hosts with this configuration:

HostA:

• HP ML150
• 5GB RAM
• 3TB for storage with HP smartArray E200i
• Intel Xeon QuadCore.

HostB:

• HP ML115 G5
• 8GB RAM
• 160GB for storage
• AMD QuadCore

Ok, lets go. I need (or I will like to do) to setup several virtual machines to accomplish different tasks (remeber, It is for personal use, like a lab environment):

• 1 virtual machine using as a DNS server and Kerberos authentication (CentOS or

RedHat)

• 2 virtual machines with RHCS installed providing several services: smtp server

(only smtp), mirror updates, squid and cifs server. (with CentOS5)

• 1 virtual machine with Windows 7 as a workstation.
• 1 virtual machine with Windows 2008 R2 server.
• 2 virtual machines with RHCS installed with OSSEC. Snort. Snortby and Splunk

server (with CentOS5 too)

• 2 virtual machines with OpenBSD firewalls with CARP and load balancing.
• 1 virtual machine as a DMZ Server.

My idea is to install DNS server (with kerberos auth) and 2 virtual virtual machines with RHCS and common services linke smtp, squid, etc onto HP ML150. And the others virtual machines running on HP ML115 server.

Where is the problem?? Problem is the storage. All storage resides on the HP ML150 server. For that reason I need to install a server as a virtual storage to run most of the virtual machines running on the server HP ML115 with the exception of firewalls and the DMZ server that resides on the HP ML115's local disk.

For backups I have an external usb disk with 1TB.

You can probably make that work if you don't care much about performance, but it would be much better to toss at least one more drive in the the ML115 - and maybe more RAM in both. Even better if you can add several drives and keep each VM that is active (the firewalls/DNS server, etc. shouldn't be busy but the squid will unless you disable the disk cache) on its own drive. And more RAM would help too.

I would probably take the memory from the 115 and put it in the 150 and have 1 highly usable system instead of a .75 and .50 usable system.

That's if I couldn't buy more memory. I would say 8GB is a min, 16GB preferred, 32GB is great. Are these single socket or dual socket?

Can the smart array be shared between two hosts?

-Ross

On Jan 31, 2011, at 12:20 PM, carlopmart carlopmart@gmail.com wrote:

On 01/31/2011 03:57 PM, Ross Walker wrote:>>> virtual machines running on HP ML115 server.

...

...

...

Where is the problem?? Problem is the storage. All storage resides on the HP ML150 server. For that reason I need to install a server as a virtual storage to run most of the virtual machines running on the server HP ML115 with the exception of firewalls and the DMZ server that resides on the HP ML115's local disk.

For backups I have an external usb disk with 1TB.

You can probably make that work if you don't care much about performance, but it would be much better to toss at least one more drive in the the ML115 - and maybe more RAM in both. Even better if you can add several drives and keep each VM that is active (the firewalls/DNS server, etc. shouldn't be busy but the squid will unless you disable the disk cache) on its own drive. And more RAM would help too.

I would probably take the memory from the 115 and put it in the 150 and have 1 highly usable system instead of a .75 and .50 usable system.

That's if I couldn't buy more memory. I would say 8GB is a min, 16GB preferred, 32GB is great

At this moment I can't buy more RAM.

. Are these single socket or dual socket?

HP ML150 is dual socket, but ML115 not.

...

Can the smart array be shared between two hosts?

No.

...

Then I suggest loading all the RAM from the 115 into the 150 (if it's the same type) and have a really good ESXi box and the 115 around as a spare in case the 150 fails.

When monies come available you could buy a second CPU or more RAM depending on what resource is needed most.

-Ross

On 1/28/2011 12:14 PM, John R Pierce wrote:

...

I need to install a virtual machine acting as a virtual storage server under

CentOS 5.x (using kvm, xen, virtualbox or vmware). This virtual storage machine needs to server storage to another ESXi server and at the same time to the host where is installed.

This is due to the limitations of hardware I have available. Both hosts needs to

server several machines.

It is very important that the virtual machine consumes the least resources

possible (host has 5GB RAM and i need to run three virtual machines minimum, including this storage server as a virtual machine).

What can be better solution: CentOS, NexentaStor, openfiler ...??

Many thaks.

no way in H*** I'd run a storage server as a VM, especially if its serving the same host as its hosted on.

storage should be rock solid infrastructure as any hiccups crash your whole environment. get a IBM DS3524 or Dell MD3200i or something for use as storage.

I don't think someone who is constrained by only having 5GB ram in a box is likely to be able to buy new storage servers...

On 01/28/11 9:19 PM, Nico Kadel-Garcia wrote:

On Fri, Jan 28, 2011 at 1:14 PM, John R Piercepierce@hogranch.com wrote:

...

no way in H*** I'd run a storage server as a VM, especially if its serving the same host as its hosted on.

*Configuration* storage, such as /etc/ files, sure. Actual backup? No virtualized host does that as well as the underlying virtualization server. There are plenty of lightweight, robust backup systems, such as rsnapshot and Amanda, which can be performanced tuned not to overwhelm the resources of the server itself and whose installation requirements are quite modest.

the OP wasn't talking about backup servers, he was talking about primary NAS/SAN storage for his two virtual servers, having a VM on one of the servers providing primary storage for both servers.