Fedora 20 Comps.xml Cheat sheet for the impaitent

The gist of this article was to teach about the comps.xml. Nowhere in the documentation does it mention that environments cannot be used in kickstart files.... so I added it.

https://fedoraproject.org/wiki/How_to_use_and_edit_comps.xml_for_package_groups#Tree.2C_Release.2C_and_Image_Composition


The difference between environments and groups is, an environment is a grouping of package groups. Groups list packages to install.
Kickstart apparently is only able to use groups.
For example if you wish to use the environment of "Minimal Install" it should be the list of package groups as such:
@standard
@guest-agents


This post and testing has led to this comment being submitted. Until this is answered correctly, it is an open issue.....

https://ask.fedoraproject.org/en/question/58555/kickstart-and-compsxml-quextion/

Now an open comps bug:

https://bugzilla.redhat.com/show_bug.cgi?id=1168062


Ever trying to make a kickstart and cant find the package set you want to use?

I know I come from the days when @base worked but it seems the world is changing and to keep up for the ones like me (the impaitent), here's a cheat sheet for Fedora 20 kickstart:

the first package i would test is 

core

but if you need something yesterday, try:

servers

or

base-system

or for the minimialist:

minimal-environment

other options you can try:

developer-workstation-environment

web-server-environment

infrastructure-server-environment

basic-desktop-environment

wget https://dl.fedoraproject.org/pub/fedora/linux/releases/20/Fedora/armhfp/os/repodata/ac802acf81ab55a0eca1fe5d1222bd15b8fab45d302dfdf4e626716d374b6a64-Fedora-20-comps.xml -O comps.xml

[root@stack1 ~]# cat comps.xml | grep "<id>"

   <id>sugar-apps</id>

   <id>admin-tools</id>

   <id>anaconda-tools</id>

   <id>lxde-apps</id>

   <id>xfce-apps</id>

   <id>arabic-support</id>

   <id>assamese-support</id>

   <id>basic-desktop</id>

   <id>web-server</id>

   <id>bengali-support</id>

   <id>bhutanese-support</id>

   <id>bodo-support</id>

   <id>c-development</id>

   <id>cinnamon-desktop</id>

   <id>core</id>

   <id>design-suite</id>

   <id>development-libs</id>

   <id>development-tools</id>

   <id>dial-up</id>

   <id>dogri-support</id>

   <id>epiphany</id>

   <id>ethiopic-support</id>

   <id>gnome-games</id>

   <id>xfce-extra-plugins</id>

   <id>eclipse</id>

   <id>fedora-packager</id>

   <id>finnish-support</id>

   <id>firefox</id>

   <id>fonts</id>

   <id>gnome-desktop</id>

   <id>gnome-software-development</id>

   <id>georgian-support</id>

   <id>greek-support</id>

   <id>guest-agents</id>

   <id>guest-desktop-agents</id>

   <id>gujarati-support</id>

   <id>hardware-support</id>

   <id>hebrew-support</id>

   <id>hindi-support</id>

   <id>input-methods</id>

   <id>japanese-support</id>

   <id>kde-desktop</id>

   <id>kde-apps</id>

   <id>kde-education</id>

   <id>kde-media</id>

   <id>kde-office</id>

   <id>kde-software-development</id>

   <id>kannada-support</id>

   <id>kashmiri-support</id>

   <id>khmer-support</id>

   <id>konkani-support</id>

   <id>korean-support</id>

   <id>kurdish-support</id>

   <id>lxde-desktop</id>

   <id>lxde-office</id>

   <id>lepcha-support</id>

   <id>libreoffice</id>

   <id>mate-desktop</id>

   <id>maithili-support</id>

   <id>malayalam-support</id>

   <id>manipuri-support</id>

   <id>marathi-support</id>

   <id>mysql</id>

   <id>mongodb</id>

   <id>multimedia</id>

   <id>lxde-media</id>

   <id>xfce-media</id>

   <id>burmese-support</id>

   <id>nepali-support</id>

   <id>oriya-support</id>

   <id>php</id>

   <id>perl-web</id>

   <id>persian-support</id>

   <id>sql-server</id>

   <id>printing</id>

   <id>punjabi-support</id>

   <id>python-web</id>

   <id>rpm-development-tools</id>

   <id>rubyonrails</id>

   <id>russian-support</id>

   <id>sanskrit-support</id>

   <id>santali-support</id>

   <id>simplified-chinese-support</id>

   <id>sindhi-support</id>

   <id>sinhala-support</id>

   <id>standard</id>

   <id>sugar-desktop</id>

   <id>tamil-support</id>

   <id>telugu-support</id>

   <id>thai-support</id>

   <id>tibetan-support</id>

   <id>traditional-chinese-support</id>

   <id>urdu-support</id>

   <id>vietnamese-support</id>

   <id>virtualization</id>

   <id>x-software-development</id>

   <id>xfce-desktop</id>

   <id>xfce-office</id>

   <id>yiddish-support</id>

   <id>base-x</id>

   <id>gnome-desktop-environment</id>

   <id>kde-desktop-environment</id>

   <id>xfce-desktop-environment</id>

   <id>apps</id>

   <id>lxde-desktop-environment</id>

   <id>cinnamon-desktop-environment</id>

   <id>mate-desktop-environment</id>

   <id>sugar-desktop-environment</id>

   <id>development</id>

   <id>servers</id>

   <id>base-system</id>

   <id>content</id>

   <id>gnome-desktop-environment</id>

   <id>kde-desktop-environment</id>

   <id>xfce-desktop-environment</id>

   <id>lxde-desktop-environment</id>

   <id>cinnamon-desktop-environment</id>

   <id>mate-desktop-environment</id>

   <id>sugar-desktop-environment</id>

   <id>developer-workstation-environment</id>

   <id>web-server-environment</id>

   <id>infrastructure-server-environment</id>

   <id>basic-desktop-environment</id>

   <id>minimal-environment</id>

tomcat 7 2nd instance sticky sessions for the impatient

In my pervious tomcat post, I quickly configured httpd and 2 tomcat 7 instances. In this post i will show you how to balance your app with sessionid's. This will allow a session



[root@web1 ~]# cat /etc/httpd/conf.d/proxy_ajp.conf

<Proxy balancer://cluster>
BalancerMember ajp://localhost:8009 loadfactor=1
BalancerMember ajp://localhost:9009 loadfactor=2
ProxySet lbmethod=bytraffic
</Proxy>
ProxyPass / balancer://cluster/ stickysession=JSESSIONID|jsessionid
ProxyPass /manager balancer://cluster/manager
ProxyPass /host-manager balancer://cluster/host-manager
ProxyPass /examples balancer://cluster/examples
ProxyPass /docs balancer://cluster/docs
ProxyPass /orbit balancer://cluster/orbit stickysession=JSESSIONID|jsessionid

Hacking the DELL PERC H700 to Support SSHD

Create the sshd's in a raid group or as separate raid 0 devices. I know this is not a performance configuration, but if you think about the chain of I/O, the RAID 0 shouldn't be your i/o bottleneck in this configuration.

I have a configuration on a PERC H700 of 5 vdisk:
(1x RAID1 Small SSD for the OS, 4x RAID0 SSHD disks for application storage).
Not a Dell supported configuration.

Tomcat 7 2nd instance for the impaitent

For the quick and dirty setup of a frontend loadbalanced (VERY BASIC) configuration.

Snag Apache.

For Fedora fans thats: 

yum install httpd
yum install mod_ssl
systemctl enable httpd.service
systemctl start httpd.service

RHEL/CentOS/OEL/etc...:

yum install httpd
yum install mod_ssl
chkconfig httpd on
service httpd start

A cheat sheet for systemd:
https://fedoraproject.org/wiki/SysVinit_to_Systemd_Cheatsheet

For Debian fans thats:

apt-get install apache2 sudo 
update-rc.d apache2 defaults 
/etc/init.d/apache2 start

Snag Tomcat.

http://tomcat.apache.org/download-70.cgi#7.0.56
Its an auto generated link based on mirrors. copy that link and paste in a wget.

wget http://apache.tradebit.com/pub/tomcat/tomcat-7/v7.0.56/bin/apache-tomcat-7.0.56.tar.gz

You will want to make two copies of tomcat for this example I am naming the instances "tomcat-1" and "tomcat-2".

wget http://apache.tradebit.com/pub/tomcat/tomcat-7/v7.0.56/bin/apache-tomcat-7.0.56.tar.gz
tar -zxvf apache-tomcat-7.0.56.tar.gz
mkdir tomcat1 tomcat2
cp -R apache-tomcat-7.0.56/* tomcat-1
cp -R apache-tomcat-7.0.56/* tomcat-2


Copy this configuration to you're tomcat-2 instance under conf/server.xml:
This file will bring up http https and ajp ports in the 9000 range (9080,9443,9009):

<?xml version='1.0' encoding='utf-8'?>
<!--
  Licensed to the Apache Software Foundation (ASF) under one or more
  contributor license agreements.  See the NOTICE file distributed with
  this work for additional information regarding copyright ownership.
  The ASF licenses this file to You under the Apache License, Version 2.0
  (the "License"); you may not use this file except in compliance with
  the License.  You may obtain a copy of the License at

      http://www.apache.org/licenses/LICENSE-2.0

  Unless required by applicable law or agreed to in writing, software
  distributed under the License is distributed on an "AS IS" BASIS,
  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
  See the License for the specific language governing permissions and
  limitations under the License.
-->
<!-- Note:  A "Server" is not itself a "Container", so you may not
     define subcomponents such as "Valves" at this level.
     Documentation at /docs/config/server.html
 -->
<Server port="9005" shutdown="SHUTDOWN">
  <!-- Security listener. Documentation at /docs/config/listeners.html
  <Listener className="org.apache.catalina.security.SecurityListener" />
  -->
  <!--APR library loader. Documentation at /docs/apr.html -->
  <Listener className="org.apache.catalina.core.AprLifecycleListener" SSLEngine="on" />
  <!--Initialize Jasper prior to webapps are loaded. Documentation at /docs/jasper-howto.html -->
  <Listener className="org.apache.catalina.core.JasperListener" />
  <!-- Prevent memory leaks due to use of particular java/javax APIs-->
  <Listener className="org.apache.catalina.core.JreMemoryLeakPreventionListener" />
  <Listener className="org.apache.catalina.mbeans.GlobalResourcesLifecycleListener" />
  <Listener className="org.apache.catalina.core.ThreadLocalLeakPreventionListener" />

  <!-- Global JNDI resources
       Documentation at /docs/jndi-resources-howto.html
  -->
  <GlobalNamingResources>
    <!-- Editable user database that can also be used by
         UserDatabaseRealm to authenticate users
    -->
    <Resource name="UserDatabase" auth="Container"
              type="org.apache.catalina.UserDatabase"
              description="User database that can be updated and saved"
              factory="org.apache.catalina.users.MemoryUserDatabaseFactory"
              pathname="conf/tomcat-users.xml" />
  </GlobalNamingResources>

  <!-- A "Service" is a collection of one or more "Connectors" that share
       a single "Container" Note:  A "Service" is not itself a "Container",
       so you may not define subcomponents such as "Valves" at this level.
       Documentation at /docs/config/service.html
   -->
  <Service name="Catalina">

    <!--The connectors can use a shared executor, you can define one or more named thread pools-->
    <!--
    <Executor name="tomcatThreadPool" namePrefix="catalina-exec-"
        maxThreads="150" minSpareThreads="4"/>
    -->


    <!-- A "Connector" represents an endpoint by which requests are received
         and responses are returned. Documentation at :
         Java HTTP Connector: /docs/config/http.html (blocking & non-blocking)
         Java AJP  Connector: /docs/config/ajp.html
         APR (HTTP/AJP) Connector: /docs/apr.html
         Define a non-SSL HTTP/1.1 Connector on port 9080
    -->
    <Connector port="9080" protocol="HTTP/1.1"
               connectionTimeout="20000"
               redirectPort="9443" />
    <!-- A "Connector" using the shared thread pool-->
    <!--
    <Connector executor="tomcatThreadPool"
               port="9080" protocol="HTTP/1.1"
               connectionTimeout="20000"
               redirectPort="9443" />
    -->
    <!-- Define a SSL HTTP/1.1 Connector on port 9443
         This connector uses the BIO implementation that requires the JSSE
         style configuration. When using the APR/native implementation, the
         OpenSSL style configuration is required as described in the APR/native
         documentation -->
    <!--
    <Connector port="9443" protocol="org.apache.coyote.http11.Http11Protocol"
               maxThreads="150" SSLEnabled="true" scheme="https" secure="true"
               clientAuth="false" sslProtocol="TLS" />
    -->

    <!-- Define an AJP 1.3 Connector on port 9009 -->
    <Connector port="9009" protocol="AJP/1.3" redirectPort="9443" />


    <!-- An Engine represents the entry point (within Catalina) that processes
         every request.  The Engine implementation for Tomcat stand alone
         analyzes the HTTP headers included with the request, and passes them
         on to the appropriate Host (virtual host).
         Documentation at /docs/config/engine.html -->

    <!-- You should set jvmRoute to support load-balancing via AJP ie :
    <Engine name="Catalina" defaultHost="localhost" jvmRoute="jvm1">
    -->
    <Engine name="Catalina" defaultHost="localhost">

      <!--For clustering, please take a look at documentation at:
          /docs/cluster-howto.html  (simple how to)
          /docs/config/cluster.html (reference documentation) -->
      <!--
      <Cluster className="org.apache.catalina.ha.tcp.SimpleTcpCluster"/>
      -->

      <!-- Use the LockOutRealm to prevent attempts to guess user passwords
           via a brute-force attack -->
      <Realm className="org.apache.catalina.realm.LockOutRealm">
        <!-- This Realm uses the UserDatabase configured in the global JNDI
             resources under the key "UserDatabase".  Any edits
             that are performed against this UserDatabase are immediately
             available for use by the Realm.  -->
        <Realm className="org.apache.catalina.realm.UserDatabaseRealm"
               resourceName="UserDatabase"/>
      </Realm>

      <Host name="localhost"  appBase="webapps"
            unpackWARs="true" autoDeploy="true">

        <!-- SingleSignOn valve, share authentication between web applications
             Documentation at: /docs/config/valve.html -->
        <!--
        <Valve className="org.apache.catalina.authenticator.SingleSignOn" />
        -->

        <!-- Access log processes all example.
             Documentation at: /docs/config/valve.html
             Note: The pattern used is equivalent to using pattern="common" -->
        <Valve className="org.apache.catalina.valves.AccessLogValve" directory="logs"
               prefix="localhost_access_log." suffix=".txt"
               pattern="%h %l %u %t &quot;%r&quot; %s %b" />

      </Host>
    </Engine>
  </Service>
</Server>


Modify your security permissions to allow manager and example apps from default tomcat.
Replace your default tomcat-users with this one under conf/tomcat-users.xml (not production recommended):


<?xml version='1.0' encoding='utf-8'?>
<!--
  Licensed to the Apache Software Foundation (ASF) under one or more
  contributor license agreements.  See the NOTICE file distributed with
  this work for additional information regarding copyright ownership.
  The ASF licenses this file to You under the Apache License, Version 2.0
  (the "License"); you may not use this file except in compliance with
  the License.  You may obtain a copy of the License at

      http://www.apache.org/licenses/LICENSE-2.0

  Unless required by applicable law or agreed to in writing, software
  distributed under the License is distributed on an "AS IS" BASIS,
  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
  See the License for the specific language governing permissions and
  limitations under the License.
-->
<tomcat-users>
<!--
  NOTE:  By default, no user is included in the "manager-gui" role required
  to operate the "/manager/html" web application.  If you wish to use this app,
  you must define such a user - the username and password are arbitrary.
-->
<!--
  NOTE:  The sample user and role entries below are wrapped in a comment
  and thus are ignored when reading this file. Do not forget to remove
  <!.. ..> that surrounds them.
-->
<!--
  <role rolename="tomcat"/>
  <role rolename="role1"/>
  <user username="tomcat" password="tomcat" roles="tomcat"/>
  <user username="both" password="tomcat" roles="tomcat,role1"/>
  <user username="role1" password="tomcat" roles="role1"/>
-->
<user name="tomcat" password="password" roles="admin-gui,manager-gui" />


</tomcat-users>


This will allow you to login to the management web application as tomcat with the password of password
Create a file in you apache configuration (conf.d) directory.
Fedora: /etc/httpd/conf.d/proxy_ajp.conf
Debian: /etc/apache2/conf.d/proxy_ajp.conf


Add this to the file:

<Proxy balancer://cluster>
BalancerMember ajp://localhost:8009 loadfactor=1
BalancerMember ajp://localhost:9009 loadfactor=2
ProxySet lbmethod=bytraffic
</Proxy>
ProxyPass /manager balancer://cluster/manager
ProxyPass /host-manager balancer://cluster/host-manager
ProxyPass /examples balancer://cluster/examples
ProxyPass /docs balancer://cluster/docs

Start apache and tomcat and test. You can test each instance, individually on http://your.servers.address:8080 and 9080. Once you have validated each instance test the webserver itself. should be something like http://your.servers.address/manager If you did it right you should see in your tomcat logs when a session hits an instance:

==> /usr/local/tomcat7-2/logs/localhost_access_log.2014-11-10.txt <==
10.1.10.1 - tomcat [10/Nov/2014:20:52:33 -0500] "GET /manager/html/list HTTP/1.1" 200 15865
10.1.10.1 - - [10/Nov/2014:20:52:33 -0500] "GET /manager/images/asf-logo.gif HTTP/1.1" 304 -
10.1.10.1 - - [10/Nov/2014:20:52:33 -0500] "GET /manager/images/tomcat.gif HTTP/1.1" 304 -
10.1.10.1 - tomcat [10/Nov/2014:20:52:33 -0500] "GET /manager/html/list HTTP/1.1" 200 15865

==> /usr/local/tomcat7-1/logs/localhost_access_log.2014-11-10.txt <==
10.1.10.1 - - [10/Nov/2014:20:52:34 -0500] "GET /manager/images/asf-logo.gif HTTP/1.1" 304 -
10.1.10.1 - - [10/Nov/2014:20:52:34 -0500] "GET /manager/images/tomcat.gif HTTP/1.1" 304 -

==> /usr/local/tomcat7-2/logs/localhost_access_log.2014-11-10.txt <==
10.1.10.1 - - [10/Nov/2014:20:52:36 -0500] "GET /manager/images/asf-logo.gif HTTP/1.1" 304 -
10.1.10.1 - - [10/Nov/2014:20:52:36 -0500] "GET /manager/images/tomcat.gif HTTP/1.1" 304 -

==> /usr/local/tomcat7-1/logs/localhost_access_log.2014-11-10.txt <==
10.1.10.1 - tomcat [10/Nov/2014:20:52:36 -0500] "GET /manager/html/list HTTP/1.1" 200 15865

==> /usr/local/tomcat7-2/logs/localhost_access_log.2014-11-10.txt <==
10.1.10.1 - tomcat [10/Nov/2014:20:52:45 -0500] "GET /manager/html/list HTTP/1.1" 200 15865
10.1.10.1 - - [10/Nov/2014:20:52:45 -0500] "GET /manager/images/asf-logo.gif HTTP/1.1" 304 -
10.1.10.1 - - [10/Nov/2014:20:52:45 -0500] "GET /manager/images/tomcat.gif HTTP/1.1" 304 -

Loadbalanced. 

  This is very rudimentary as it has no session awareness. That my friend, is a 5 minute session for another day.......



Openstack Architecture: The power and brief cooling for this project.

I will get down to power configurations more when the budget grows but I have chosen high voltage 240V AC with an L6-20R to L6-20P 20A 208V commonly available. The power poles are 20 port c13 - c14 and 4 port c19 - c20. This higher current leads to lower amperage meaning more servers can run in your rack. Be careful to know you BTU and wattage calculations as well to prepare your power for your devices and and appropriate cooling. Cooling info is sometimes as btu by the vendor but if you do not know your btu, you can find it with a simple calculation:

amps x volts = watts 

watts x 3.41 = BTU/hour

max output for an R610 = 2,968.6 BTU/hour

I am using in this configuration Dell R610 series servers as I was able to get a bunch of them cheaply with a good amount of memory inside. I'll explain the hardware in another post.

Taken from http://teamsilverback.com/knowledge-base/data-center-power-selecting-the-right-voltage/ and a much better reference to explain why to choose high voltage and how to calculate cooling as well.

"

What Voltage Should I Use? 120V, 208V, 240V, AC or DC?

Choosing the right voltage standard for data center design will have a significant impact on power utilization efficiency, cost of implementation, TCO, cooling efficiency, and space utilization. Specifically in North America, 120V Single Phase AC power is often used by tradition over sensibility and is dramatically less efficient and more costly than higher voltage options.

Most modern information technology equipment is designed to operate at 100V to 250V to accommodate North American 120/208V, Japanese 100/200V, and 230V used in the rest of the world to allow for worldwide power compatibility. Without going into confusing power calculations and details (readily available on the web if interested) using the highest voltage available is widely regarded as the most efficient choice striving to reach the 85-90% efficiency range. The emerging standards recommendation for data centers in North America to move to 240V power significantly improves efficiency over 120V distribution, requires less wiring, uses smaller step down transformers, and is less costly to implement.

North American UPS standard output power of 277/480V can be stepped down to 240V with a single-winding autotransformer. For 120/208V power distribution, an isolation transformer is required with two windings, primary and secondary. The single-winding autotransformer for 240V power is typically 90% smaller and less costly than the PDU isolation transformers required for step down to 120/208V.

While some advocate using DC over AC power citing power efficiency benefits, the availability of DC power delivery systems outside of telecom equipment racks is not widely supported and the proposed efficiency gains are similar to the gains achieved by operating at 240V AC over 120/208V AC. Thus, standardizing on 240V AC power can achieve similar benefits at a much lower cost, with less complexity, and provide greater overall compatibility.

Top 10 Reasons to Use 240V Power over 120V/208V

1. 277/480V to 240/415V step down autotransformer is 90% smaller and less expensive than 120/208V PDU isolation transformer
2. Elimination of large PDU transformers reduces cooling costs and requires less space
3. 240V distribution yields the same power capacity with nearly half the current as 120V
4. Fewer circuit breakers required – Reduces points of failure
5. Fewer branch circuits to racks – Reduces wiring cost, weight, and bulk
6. Less wiring bulk increases airflow and cooling efficiency, decreases cable weight
7. Branch circuit conductor wires for 120V distribution are 2-3 times larger than 240V yielding the same power capacity
8. Using higher voltage at lower amps for the same power yield allows for use of smaller conductor wires thus reducing capital costs and reducing line lost due to resistance
9. Nearly every other country outside of North America is already operating at similar 230V distribution – worldwide consistency has many practical benefits

10. Moving from 120V to 240V increases efficiency by 2-3.5% per server – accumulated over 100’s of servers this adds up to significant energy savings and greater capacity "

The reasoning behind power poles with c20 is simple. ANY ups with anything to c19 is fairly easy to find to connect the entire power pole to UPS power and maintain a single rack configuration.

Openstack Architecture: The View.

To build a reasonably priced, single in rack or rackable cloud service to deploy in multiple locations to quickly bring a company into the benefits of cloud architecture and leverage the MASSIVE benefits of openstack storage Swift, cheers and beers for the guys writing all parts of openstack, I hope to catch up on the code soon and join.
This will be a growing blog of different designs that are already running, and how they get enveloped into an openstack configuration. Will be doing tests and configurations of various conceptual  node configurations. The architectures will go briefly over these concepts:

Power and Cooling

Physical Devices List (All Cabling and Devices)

Physical Network Architecture to Support Openstack (Cabling and Network)

Physical Servers Configuration

Openstack Architectures

Openstack Layout