ss12015 Managing Operstack InlecraiDatafExtecral network n ane interface | Free and Open Source Software Knowledge Base Free and Open Source Software Knowledge Base Managing Openstack Internal/Data/External network in one interface A common problem for people who want to try Openstack without a full blown hardware setup is that they have just one network interface. Openstack identifies three distinct networks Internal Network This is where all your inter process communication happens. This is where your mysql- server/queue-server etc are listening and this is where your services exchange information among themselves. On a proper set up this network should be isolated and secured and the interface connected to this network should not be added to any bridge. Data Network This is where your Instances talk to each other and to their network's 13 and dhep services. This network again should be data networks are mapped to a physical networks which will be available for neutron to use using config file parameters. It is the physical network that you denote as ‘provider:physical_network’ docs.openstack.org/api/openstack- ‘lated and secured. There can be more than one data network. The in the ‘neutron net-create (http: -epetsskb wordpress.com/20"406Omanaging-openstac-niernaldeinexteral-netwotk-in-one-inerface! 1138xsva0s Managirg Operetck eral atatxteral network in oe irerface| Free and Open Source Sofware Knowledge Base network/2.0/content/provider_network_create.html)’ APT call. You need not worry about choosing the physical_network for each network you create as neutron will choose it for you if you did not. bridge_mappings Present inside the ‘ovs’ section of ‘ml2_conf.ini’. You tell Neutron which physical networks are available for use through this parameter. You also have to tell which bridge to use in order to reach that physical network. Thus ‘bridge_mappings’ is a comma separated list of ‘physical_network:bridge_name’ pairs. You also have to make sure the bridges that you mapped to physical networks exists on the host. flat_networks Present under ‘ml2_type_flat’ section. Configured in case of flat networks. This separated list of physical networks that are flat(no vlan involved) just a comma network_vlan_ranges Present under ‘mI2_type_vlan’ section. Configured in case of vlan networks. This is similar to flat_networks except that for each physical networks there is a start and an end vlan appended with a ‘:’ between them. local_ip Present under ‘ovs’ section’. In case you are using GRE mode this parameter will tell neutron which IP to bind and run GRE on. This in turn determines which interface and network should be used as data network. So it is a good idea to use an interface other than the one used for internal network. Finally, unless you are using ‘GRE’ alone, you have to add one of the host’s network interface to every bridge specified so that all physical networks are now bridged to their corresponding data network. Using a little trick you can even map more than one physical network to a data network. ps:tosskb wordpress. comn/20"406 1!managing-openslack:-nieralataexteral-network-ir-on-inerface! 2198ss12015 Managing Operstack IneralDatalExterral network in ane interface | Free and Open Source Software Knowledge Base External Network This network is used for two purposes. 1. To expose the services(nova-api, glance-api etc) to consumers outside of Openstack. 2. To allow your Instances to be accessible from outside of Openstack, through floating-ip. Tt is a good idea to use two external networks for the above two purposes. That way you can restrict all ports other than those on which your exposed services are listening. In Neutron and external network is one on which you have ‘router:external’ set to true. Only then can you create foating-ips on it. In all other ways all rules that apply to physical networks also apply here. Normally you would want to chose a flat physical network for creating external network. Otherwise you would have to ask your network administrator to set up vlan on the switch port connecting to the machine running your 13-agent and things start to get ugly. The host interface connecting to the external network should not have any form of security. You should allow security groups to do that job. Network Node Controller Compute hep 8 metadata = 2 ann I I I br-int brint in-brotht ini-brox int-br-etht > phy-bretni| | phy-brex phy-br-eiht [brett br-ex bretht [etna] Full blown OpenStack setup Using the same Interface for all Networks ps:tosskb wordpress. comn/20"406 1!managing-openslack:-nieralataexteral-network-ir-on-inerface! 436ss12015 Managing Operstack InlecraiDatafExtecral network n ane interface | Free and Open Source Software Knowledge Base Finally we arrive at the purpose of this blog. This blog gives you plenty of information and reason why you should not do this but while you are experimenting all is fair. Assumptions: eth0 is the only available port 1. 2. bridge_mappings-Physnetl :br-etht, External_network:br-ex 3. network_vlan_ranges=Physnetl:100:200 4, flat_networks-External_network ovs-vsct1 add-br br-etho ovs-vsctl add-port br-eth@ ethe ifconfig br-eth® up ip link set br-eth@ promisc on ip link add proxy-br-eth1 type veth peer name eth1-br-proxy ip link add proxy-br-ex type veth peer name ex-br-proxy ovs-vsctl add-br br-eth1 ovs-vsctl add-br br-ex ovs-vsctl add-port br-eth1 eth1-br-proxy ovs-vsctl add-port br-ex ex-br-proxy ovs-vsct1 add-port br-eth® proxy-br-eth1 ovs-vsct] add-port br-eth@ proxy-br-ex ip link set ethi-br-proxy up promisc on ip link set ex-br-proxy up promisc on ip link set proxy-br-eth1 up promisc on ip link set proxy-br-ex up promise on 1. What we have done is added a new bridge br-eth0 and added eth0 to it. 2. Assign eth0’s ip address to br-eth0 and set the interface in promiscuous mode. 3. Then we create two veth pairs. In case you are not aware they are like virtual cables. 4, We connect br-ethi and br-ex to br-eth0 using the veth pairs. 5. Then we enable promiscuous mode and bring up all the interfaces we use. ps:tosskb wordpress. comn/20"406 1!managing-openslack:-nieralataexteral-network-ir-on-inerface! 6ss12015 Managing Operstack IneralDatalExterral network in ane interface | Free and Open Source Software Knowledge Base Network/Compute/Controller Node dhcp 13 metadata vm vm wi service || service | | “service [ [ [ br-int int-br-ex phy-br-ex br-ex br-proxy Single machine setup with 1 interface Running Controller and Network on same host Sometimes It is desired to have controller and Network node running on same machine and the machines have only two network interfaces each. The compute node requires only two interfaces s shown in the picture below. However in the network node we can combine the internal and external network by adding eth0 to br-ex and assigning br-ex with the ip address of eth0. ovs-vsct1 add-port br-ex etho ifconfig br-ex up ip link set eth up promisc on ps:tosskb wordpress. comn/20"406 1!managing-openslack:-nieralataexteral-network-ir-on-inerface! 98ss12015 Managing Operstack IneralDatalExterral network in ane interface | Free and Open Source Software Knowledge Base Network/Controller Node Compute dhep 13. metadata service || service service a i a i I L br-int br-int int-br-ex int-br-etht int-br-etht phy-br-ex phy-br-eth1) phy-br-eth1 br-ex br-eth1 br-eth1 2 Machine with 2 interfaces each If both your servers have only a single nic you may follow the below setup. On network/controller node ps:tosskb wordpress. comn/20"406 1!managing-openslack:-nieralataexteral-network-ir-on-inerface!ss12015 Managing Operstack InlecraiDatafExtecral network n ane interface | Free and Open Source Software Knowledge Base #add all bridges ovs-vsctl add-br br-int ovs-vsct1 add-br br-ex ovs-vsct1 add-br br-eth1 ovs-vsct1 add-br br-proxy #Create Veth pairs ip link add proxy-br-eth1 type veth peer name eth1-br-proxy ip link add proxy-br-ex type veth peer name ex-br-proxy #attach bridges using veth pair ovs-vsctl add-port br-eth1 eth1-br-proxy ovs-vsctl add-port br-ex ex-br-proxy ovs-vsct1 add-port br-proxy proxy-br-eth1 ovs-vsct1 add-port br-proxy proxy-br-ex Assign eth@'s ip address to br-proxy ifconfig br-proxy up #Bring up the interfaces ip link set ethi-br-proxy up promise on ip link set ex-br-proxy up promisc on ip link set proxy-br-eth1 up promisc on ip link set proxy-br-ex up promisc on On the Compute node ovs-vsct1 add-br br-eth1 ovs-vsctl add-port br-eth1 etha Assign ethe's ip addres to br-eth1 ifconfig br-eth1 up #Bring up the interfaces ip link set eth@ up promisc on The pictorial representation would be something like below ps:tosskb wordpress. comn/20"406 1!managing-openslack:-nieralataexteral-network-ir-on-inerface! 136ss12015 Managing Opsnstack InecralDatalExterral network n one interface | Free and Open Source Software Knowledge Base Network/Controller Node dhep 13 | metadata service Compute “ww br-int int-br-eth1 | service service phy-br-eth br-eth1 Dual machine setup with single nic each ‘About these ads (ntip/Wwordpress.com/about- _ © = 10/06/2014 @ Akilesh these-ads/) 101 thoughts on “Managing Openstack Internal/Data/External network in one interface” 1. Tomasz says: 11/11/2014 at 8:15 PM Hi Akilesh. Thanks for the great post. Based on the discussion below this post seems that OpenStack community really needs solutions other than 3 nodes with 3 NICs (as described on doc.openstack.org installation instructions). T’'m new to Openstack and I’m trying to create a “sandbox” setup made of 2 machines (1 Controller/Network machine, 1 Compute machine). I’m still tryng to figure out how to configure neutron so i have few questions here. 1. The description for 2 Machines/ 2 NICs states that I should create br-ex and add eth0 to it. It doesn’t say anything about br-eth1 although the figure shows that I should create one. Is this correct? hepfoskt werdres.con/20"406/1kmanging-openstack-inuraldiextea-rewotki-oerartaca! a6xsva0s Managirg Operetck eral atatxteral network in oe irerface| Free and Open Source Sofware Knowledge Base 2. Inmy solution the management/external network is a LAB network which already has a DHCP server (I setup data network as a closed network between Controller/Network node and compute node) Is it possible to configure the “Floating” network in a way that VMs will get DHCP from the LAB network via bridge and not from neutron? Tl be very grateful for help Thank You Tomasz Reply © Akilesh says: 12/11/2014 at 6:08 PM This post is for first time users, trying to get a taste of openstack using whatever little hardware they have. The official documentation include information needed for building a full blown infrastructure. 1. br-eth1 is also necessary. In fact all bridges that you have configured in ‘ml2_conf. should be created. 2. floating ip should be created and used by OpenStack users as and when required and only Neutron can allocate the same. Neutron will allocate floatingip from the subnet created for the external network. In case you share the external network with other devices that are being managed by dhcp server then you can use allocation pool option when creating subnet on external network. That way you can use a subset of the ip address’ on external network for openstack, while the remaining can be used by other devices on your network. Be sure to read through OpenStack documentation in case you later want a production [] network. They have lot of information regarding it. Reply 2. Vinay says: 31/10/2014 at 1:42 AM Thave used single node and single NIC for setup. Now I can not ssh to single node. Reply © Vinay says 31/10/2014 at 1:51 AM Here is my configuration done on Single node and single NIC. rootav: ~# ovs-vsctl add-br br-ethO rootev: -# ovs-vsctl add-port br-eth0 em1 rootey: “# ifconfig br-eth0 10.99.14.14 up rootav: :# ip link set br-eth0 promisc on rootey: -# ip link add proxy-br-ethl type veth peer name eth1-br-proxy rootey: -# ip link add proxy-br-ex type veth peer name ex-br-proxy rootey: ~# ovs-vsetl add-br br-eth1 root@vsa-icehouse01:-# ovs-vsctl add-br br-ex root@vsa-icchouse01:~# ovs-vsctl add-port br-ethl eth1-br-proxy ps:tosskb wordpress. comn/20"406 1!managing-openslack:-nieralataexteral-network-ir-on-inerface! 936ss12015 Managing Opestack hieralDataExteral retwerkin oe interface | Free ar Open Source Sofware Knowledge Base root@vsa-icehouse1:~# vi xx root@vsa-icehouse01:~# ovs-vsctl add-port br-eth! eth1-br-proxy ovs-vsetl: cannot create a port named eth1-br-proxy because a port named eth1-br-proxy already exists on bridge br-eth! root@vsa-icehouse01:~# bash -x xx ctl add-port br-eth1 eth1-br-proxy ctl: cannot create a port named eth1-br-proxy because a port named eth1-br-proxy already exists on bridge br-eth1 + ovs-vsetl add-port br-ex ex-br-proxy ‘ctl add-port br-eth0 proxy-br-eth1 .ctl add-port br-eth0 proxy-br-ex + ip link set eth1-br-proxy up promisc on + ip link set ex-br-proxy up promisc on + ip link set proxy-br-eth1 up promisc on + ip link set proxy-br-ex up promisc on root@vsa-icehouse01:~# cp /etc/neutron/metadata_agent.ini /etc/neutron/metadata_agent.ini.ORIG root@vsa-icehouse01:~# vi /etc/neutron/metadata_agent.ini root@y: :~# cp /etc/neutron/dhep_agent.ini /etc/neutron/dhep_agent.ini.ORIG rootev: -# vi /etc/neutron/dhep_agent.ini root: .-# vi /etc/neutron/dhep_agent.ini root@vs :-# cp /etc/neutron/13_agent.ini /etc/neutron/13_agent.ini.ORIG rootev: :~# vi /etc/neutron/13_agent.ini root@vsa-icehouse01:~# service neutron-server restart; service neutron-plugin- openvswitch-agent restart;service neutron-metadata-agent restart; service neutron-dhep- agent restart; service neutron-I3-agent restart neutron-server stop/waiting neutron-server start/running, process 18947 neutron-plugin-openvswitch-agent stop/waiting neutron-plugin-openvswitch-agent start/running, process 18963 neutron-metadata-agent stop/waiting neutron-metadata-agent start/running, process 18976 neutron-dhcp-agent stop/waiting neutron-dhcp-agent start/running, process 18999 stop: Unknown instance: neutron-I3-agent start/running, process 19021 ‘a-icehouse01:~# neutron agent-list a4 lid | agent_type | host | alive | admin_state_up | + —- a4 ---+ | 23d4174f-cbel-4042-be04-bd5alfdeb7aa | Open vSwitch agent | vsa-icehouse01 |__| True | | 3773d2a1-a0bb-4edd-af21-0b38e242280f | L3 agent | vsa-icehouseO1 |__| True | | 49ced6a0-0049-454b-a0c6-e3eb6568173 | DHCP agent | vsa-icehouse01 |__| True | ps:tosskb wordpress. comn/20"406 1!managing-openslack:-nieralataexteral-network-ir-on-inerface! 106ss12015 Managing Operstack IneralDatalExterral network in ane interface | Free and Open Source Software Knowledge Base | 729a9360-f3aa-4916-a7cc-cb252679/019 | Metadata agent | vsa-icehouse01 |__| True | root@vsa-icehouse0) Re © Vinay says: 01/11/2014 at 1:52 AM after updating /etc/network/interfaces file as below and rebooted , it looks working. got info from http://zcentric.com/2014/07/07/openvswitch-kvm-libvirt-ubuntu-vlans- the-right-way/ root@vsa-icehouse01:~# cat /etc/network/interfaces # This file describes the network interfaces available on your system # and how to activate them. For more information, see interfaces(5). # The loopback network interface auto lo iface lo inet loopback # The primary network interface auto em1 iface em1 inet manual ovs_bridge br-ethO ovs_type OVSPort adress 0.0.0.0 # address 10.99.14.14 # netmask 255.255.255.0 # network 10.99.14.0 # broadcast 10.99.14.255 # gateway 10.99.14.1 # # dns-* options are implemented by the resolveonf package, if installed # dns-nameservers 15.226.142.15 # dns-search tplab.tippingpoint.com # The bridgr interface auto br-eth0 iface br-eth0 inet static address 10.99.14.14 netmask 255,255,255. network 10.99.14.0 broadcast 10.99.14.255 gateway 10.99.14.1 # dns-* options are implemented by the resolvconf package, if installed dns-nameservers 15.226.142.15 dns-search tplab.tippingpoint.com ovs_type OVSBridge ps:tosskb wordpress. comn/20"406 1!managing-openslack:-nieralataexteral-network-ir-on-inerface! v8xsva0s Managirg Operetck eral atatxteral network in oe irerface| Free and Open Source Sofware Knowledge Base ovs_ports br-eth0 bridgr_porte em1 bridge_stp off bridge_fd 0 bridge_maxwait 0 root@vsa-icehouse01:~# © akilesh1597 says: 01/11/2014 at 3:06 PM Thank you Vinay for sharing. As I had mentioned in the post your host will not be able to receive any data on an interface that was added to any ovs bridge. That is why assigning ip address to the bridge is necessary. You would not be able to do it(you will loose ssh access as soon as you add the interface to the bridge) unless you had direct keyboard access to the host. Setting the configuration in the interfaces file is the permanent solution. © dashesy says: 03/01/2015 at 7:49 AM Iget error ‘eth1-br-proxy’ device not found. Is there any distro-agnostic way to set up that bridge? I asked also [here] © Akilesh says: 05/01/2015 at 12:39 PM the instructions should work on any distro. ‘eth1-br-proxy’ is a veth device. You have to create it before using it. The relevant command is ‘ip link add proxy-br-eth1 type veth peer name ethl-br-proxy’. May be you missed that step. © Anshuman says: 25/01/2015 at 10:08 PM Hi, I'm having a similar issue. My eth0 is 192.168.1.219. I have followed the tutorial by creating all the bridges and interfaces. What should be the subnet value of the network which runs the VM. Ihave chosen a value of 192.168.1.0/24 but my host machine is not able to ping the VM. © Akilesh says 02/02/2015 at 6:37 PM The host can not ping the vm directly. Do you mean host is not able to reach vm's floatingip? 3. pawansolanki says: 27/10/2014 at 10:55 PM ps:tosskb wordpress. comn/20"406 1!managing-openslack:-nieralataexteral-network-ir-on-inerface! v8xsva0s Managirg Operetck eral atatxteral network in oe irerface| Free and Open Source Sofware Knowledge Base Reblogged this on pawansolanki and commented: Can we launch a instance with two vnic card and assign ip to them seperately Reply © Johnson D says 28/10/2014 at 1:26 PM While launching the instance you can select two networks and two nics will be added by nova, one each in a network. If you want to add interfaces at run time check the ‘update- server’ command of the nova api reference. Reply 4, jing says: 23/10/2014 at 2:14 PM Hi, thanks for the detail post. I tried to setup the environment on a single node with 1 NIC. Tcan create the instances but the network still have some problems and I have some questions. 1. created the br-ex, br-ethl, br-int, but how are they been used? I mean who will use br- eth1/br-ex/br-int and how do they use it? I can only find some configurations like ” Jneutron/plugins/ml2/ml2_conf.ini:bridge_mappings=External:br-ex, Intnetl :br-eth1 /neutron/I3_agent inizexternal_network_bridge = br-ex 2. my external network is through dhep, how to handle this? It seems that the floating IPs are not assigned by the external dhcp server. Please help on this, thanks:) 26/10/2014 at 8:02 PM These bridges are used by neutron to ensure connectivity between your instances and various neutron services like the I3-agent, dhcp-agent etc. These details are explained in this post. floatingips are not allocated by your dhep server in external network. The floatingips are allocated on the external network and assigned to instances as and when you create them using floatingip-create command. This is explained in this post , towards the end of the post. Reply © jing say: 27/10/2014 at 11:28 AM hi, where are the detailed explanations? _ the links are miss: Idid a grep in /etc and found something below. In I3_agent.ini, external_network_bridge = br-ex. In dhep_agent.ini, ovs_integration_bridge-br-int(I guess this is the default value?) But I didn’t find anything about br-eth1 except in ml_conf.ini, under [ovs] there’s a bridge_mappings, which specifies br-eth1 as vlan. Is there anything else? ps:tosskb wordpress. comn/20"406 1!managing-openslack:-nieralataexteral-network-ir-on-inerface! 1988ss12015 Managirg Operetck eral atatxteral network in oe irerface| Free and Open Source Sofware Knowledge Base For the floating IP, if they are not allocated by the dhep server, then how can they be visited by others from external network, a little confused about this. I think each instance will have an internal address, and it is mapped with some external “global” ip address through IPtable, right? Thanks © akilesh1597 says: 27/10/2014 at 11:44 AM Sorry missed the links. I have edited the my previous reply. Kindly read through both the links. [hope at the end you might understand how this works. If not we shall discuss further. © jing says: 28/10/2014 at 12:48 PM hiakilesh Thanks for the great posts, I think it solved some of my questions and raises some others:) In config file ml2_conf.ini, under [mI2_type_flat], flat_networks-External. Here the “External” is just a general name or it should match the ‘physical_network’ in TABLE ml2_network_segments? I tried to use Horizon to create the external network, it will set network_type to vlan and physical_network to Intnetl by default. So do I have to use command line to create it manually with provider:physical_network=External and providernetwork_type=flat? Thanks © Johnson D says 28/10/2014 at 1:17 PM The networks(both flat_networks and vlan_networks) that you define in ‘ml2_conf.ini’ are called physical networks. When you create a network using ‘neutron net-create’ or using horizon, visualize it as these networks will be created on top of these physical networks. These are exactly what you find in database. Each network in openstack would be mapped to a physical network. For you second que: options. ion, you are correct you have to use the command line for those © jing say: 28/10/2014 at 2:13 PM hiJohnson| ‘My question is whether the names(External, physnetl, physnet2, etc..) in ml2_conf.ini should be the same as the ones created with neutron net-create, what if a tenant create the same physical_network name as the provider physical network? (for example, I want to use flat for external networks and vlan for tenant networks), if their names are the same, it will cause some trouble, right? ps:tosskb wordpress. comn/20"406 1!managing-openslack:-nieralataexteral-network-ir-on-inerface! aness12015 ° Managirg Operetck eral atatxteral network in oe irerface| Free and Open Source Sofware Knowledge Base © Johnson D says 28/10/2014 at 3:25 PM That configurations is not at all possible. Each physical network should be unique, whether vlan or flat. jing says: 29/10/2014 at 7:19 AM hiJohnson| In my opinion, administrator will create the provider network while the tenant will create the tenant network. As a tenant, he may not aware what the provider network “physical_network” name is, maybe they both choose the same name, or there’s something Imisunderstand here? Reply © akilesh1597 say: 29/10/2014 at 12:09 PM You need not specify the provider:physical_network for each network you create. Neutron will do it for you. The administrator alone will specify the parameter explicitly to create some shared networks. Even if a user is trying to use something which has already been assigned to another user, neutron is smart enough to throw errors. You can try creating two network on a single flat physical network and check for yourselves. If however you do the same on a vlan network it would be created but the networks will have different segmentation id. If you explicitly specify the segmentation id that has already been used, neutron will again throw error. © jing says: 29/10/2014 at 1:53 PM Thanks akilesh, I missed the segmentation id. Now I have followed the steps in this post(at least I though I had), when starting a new instance, it will not be able to get the metadata(some address 169.254.169.254) The qdhep and qrouter can ping each other and instance can get an ip address from qdhep(from the log), but I can’t ping the instance in qdhep or qrouter.(At that time the instance is stucking trying to get metadata). Any idea on what’s wrong of this? What may cause an instance not able to ping the router? © akilesh1597 say: 29/10/2014 at 4:06 PM Ido not understand what you mean by ‘qdhcp and qrouter can ping each other’. Tell me 1. Have you created a router and attached the instanci 2. Can the instance ping the router? subnet to the router? © jing say: 30/10/2014 at 6:33 AM ps:tosskb wordpress. comn/20"406 1!managing-openslack:-nieralataexteral-network-ir-on-inerface! wwe8ss12015 Managirg Operetck eral atatxteral network in oe irerface| Free and Open Source Sofware Knowledge Base Sorry I didn’t express myself clearly. T've created the router and the instance can’t ping it. From the host machine with “ip netns list” I can see 2 netns qrouter-xxx and qdhep-xxx With “ip netns exec qrouter-xxx ifconfig”, there are 3 interfaces, lo, qg-xxx(for externel network IP 10.239.67.6) and qr-xxx(internal network 10.0.1.1), With “ip netns exec qdhep-xxx ifconfig” there are 2 interfaces, one is lo and the other is the internal dhep address(10.0.1.101, I set the subnet allocation pool from 10.0.1.100 to 10.0.1.120) For “qdhep and qrouter can ping each other” I mean if I enter the qdhep namespace I can ping qrouter’s internal ip addr (ip netns exec qdhcp-xxx ping 10.0.1.1 is OK and ip netns exec qrouter-xxx ping 10.0.1.101 is also OK). But neither can ping 10.0.1.106, which is the ip address of the instance. And in the console instance, it can’t ping 10.0.1.101 or 10.0.1 There must be something wrong but I don’t know where to check © akilesh1597 say’ 01/11/2014 at 4:12 PM A little correction. The qrouter-xxx and qdhep-xxx are namespaces. They are used to have different isolated network stack on the same host machine. The qrouter-xxx namespace holds interfaces of the router you create, while the qdhcp-xxx namespace holds the interface to which your dhep server(dnsmasq) attaches to. Further a lot of people face this issue and I am unable to answer them just because the number of caus for this could be too many and probing each one and explaining it to the users is a never ending process. I'll try my best on this one though. © jing says: 30/10/2014 at 8:33 AM something more. The instance seems can get the internal ip at boot time(from the log). Then it will stuck a long time trying to get metadata and failed. Then I enter the instance, do a ifdown and ifup, it will not be able to get an IP anymore. Is it caused by some iptable rules for qbr which connects the tap of instances and the qvb? Ichecked the iptable rules of qbr, don’t find anything suspicious. -A neutron-filter-top -j neutron-openvswi-local -A neutron-openvswi-FORWARD -m physdev ~physdev-out tap76dfb669-5e —physdev- is-bridged -j neutron-openvswi-sg-chain -A neutron-openvswi-FORWARD -m physdev ~physdev-in tap76dfb669-5e -physdi is-bridged -j neutron-openvswi-sg-chain -A neutron-openvswi-INPUT -m physdev ~physdev-in tap76db669-5e —physdev-is- bridged -j neutron-openvswi-076dfb669- -A neutron-openvswi-i76dfb669-5 -m state -state INVALID -j DROP -A neutron-openvswi-i76db669-5 -m state state RELATED, ESTABLISHED -j RETURN -A neutron-openvswi-i76dfb669-5 -s 10.0.1.101/32 -p udp -m udp sport 67 ~dport 68 -j ps:tosskb wordpress. comn/20"4061Gimanaging-openslack:-niernalataexeral-network-ir-on-inerface! 18ss12015 Managirg Operetck eral atatxteral network in oe irerface| Free and Open Source Sofware Knowledge Base RETURN -A neutron-openvswi-i76dfb669-5 -j neutron-openvswi-sg-fallback -A neutron-openvswi-076dfb669-5 -p udp -m udp -sport 68 -dport 67 -j RETURN -A neutron-openvswi-076dfb669-5 -j neutron-openvswi-s76dfb669-5 -A neutron-openvswi-076dfb669-5 -p udp -m udp -sport 67 —dport 68 -j DROP -A neutron-openvswi-076dfb669-5 -m state -state INVALID -j DROP -A neutron-openvswi-076dfb669-5 -m state ~state RELATED, ESTABLISHED RETURN -A neutron-openvswi-076dfb669-5 -j RETURN -A neutron-openvswi-076dfb669-5 -j neutron-openvswi-sg-fallback -A neutron-openvswi-s76dfb669-5 -s 10.0.1.106/32 -m mac -mac-source FA:16:3E:88:AE:EA -j RETURN -A neutron-openvswi-s76dfb669-5 -j DROP -A neutron-openvswi-sg-chain -m physdev ~physdev-out tap76dfb669-5e physdev-is- bridged -j neutron-openvswi-i76dfb669-5 -A neutron-openvswi-sg-chain -m physdev —physde bridged -j neutron-openvswi-076dfb669-5 -A neutron-openvswi-sg-chain -j ACCEPT -A neutron-openvswi-sg-fallback -j DROP -A nova-api-INPUT -d 10.239.67.77/32 -p tcp -m tep ~dport 8775 -j ACCEPT © akilesh1597 says: 01/11/2014 at 4:36 PM A few questions. Is this the iptable rule of the host machine on the default namespace? , What distro are you using? Which document you used to install openstack? and Please tell me you are not using nova-network. © jing says: 03/11/2014 at 6:56 AM the iptable rules are on the default namespace, and I’m using ubuntu 14.04 following I'm using neutron, not the legacy nova-network. © jing says: 04/11/2014 at 8:07 AM Tthink I found the root cause. The system had also installed docker, which will create a VNIC and a bunch of routing rules, I think it somehow affect the openstack. Also I have always set http_proxy and https_proxy, which will have trouble while getting metadata. After purge docker and unset http_proxy, I can get the internal IP now. Thanks! © akilesh1597 says: 04/11/2014 at 11:17 AM ps:tosskb wordpress. comn/20"406 1!managing-openslack:-nieralataexteral-network-ir-on-inerface! 8ss12015 Managing Operstack InlecraiDatafExtecral network n ane interface | Free and Open Source Software Knowledge Base Great find. As you see even I am still leaning from user's comments . Keep your findings documented somewhere. © akilesh1597 says: 04/11/2014 at 11:19 AM Also I am planning for a debugging guide shortly with all possible root causes. Your experience will surely be documented there. Thanks for the info. 5. Prashant says: 17/10/2014 at 6:04 AM Thave one NIC card on Network and Compute node. | installed openstack Ubuntul4.04 Icehouse. Can you help me how to create internal and external interfaces using one NIC. Iam confused after reading all the posts here. Reply © akilesh1597 say’ 21/10/2014 at 7:19 PM Hi, The post has both commands and supporting pictorial representations of the setup. Beyond these I will not be able to provide any direct support to you as of now. If you have trouble understanding any part of the post kindly let me know what your exact doubt is. I can clarify. Reply 6. kashin08 say: 22/09/2014 at 8:48 AM Sorry that it seems my network problem is out of focus... Thave updated the experiment result on 9/9/2014. Can it help to dig into the problem and anything else i can help to find the root cause? Reply © akilesh1597 say’ 22/09/2014 at 5:31 PM Sorry for the delay. I assume eth0 is in promiscous(since you say the packets reach br- eth0(in my case br-proxy)). If you read my other post on 13 I have explained in detail what steps are taken my neutron 13 agent to do natting of floatingip to instance’s private ip. I hope you read through it. Launching instances in external network is not the correct thing to do(Although It might work in case of single machine setup alone). Further from you result the only possible cause is ‘ip forwarding is not enabled in your machine’. Please check ‘https://fosskb. wordpress.com/2014/06/25/a-bite-of-virtual-linux-networking/’ under section ‘ip forwarding’. If you have enabled but still have problems post the output of ‘ip netns exec qrouter- iptables -t nat -nvL’ and ‘ip tables exec qrouter- ip link show/ip addr show’. Reply © kashin08 says: ps:tosskb wordpress. comn/20"406 1Gimanaging-openslack:-nieralataexteral-network-ir-on-inerface! wwexsva0s Managirg Operetck eral atatxteral network in oe irerface| Free and Open Source Sofware Knowledge Base 26/09/2014 at 6:45 AM Yes, the ip forwarding is enabled (checked from /etc/sysctl.conf & cat /proc/sys/net/ipv4/ip_forward). The result of “iptables & ip link/addr show” are attached in below link. ht 0 www ,dropbox.com/s/xpscchu7hgm05an/result.los © akilesh1597 says: 29/09/2014 at 6:55 PM Hi. I viewed your logs and there is something seriously wrong with the output of ‘ip netns exec qrouter- iptables -t nat -nvL’ that you posted. Below is the excerpt. Chain neutron-13-agent-PREROUTING (1 references) pkts bytes target prot opt in out source destination 2@ 120@ REDIRECT tcp -- * * @.0.0.0/@ 169.254.169.254 tcp dpt:8@ redir ports 9697 1 84 DNAT all -- * * 0.0.0.0/@ 192.168.1.101 to:192.168.2.12 Chain neutron-13-agent-float-snat (1 references) pkts bytes target prot opt in out source destination 21 1764 SNAT all -- * * 192.168.3.10 0.0.0.0/0 to:192.168.1.101 Thave highlighted the problem. The nat happens from 192.168.1.101 to 192.168.2.12 during prerouting phase, which is what happens to incomming packets, whereas the outgoing packets are natted from 192.168.3.10 to 192.168.1.101 which is entirely different. I am still unsure how this error crept in or if this was a copy paste error when you posted the log. Please stop the neutron-I3-agent, then clear all iptable rules inside the router namespace and start the neutron-13-agent again. service neutron-13-agent stop ip netns qrouter- iptables --flush service neutron-13-agent start Check the rules again and let me know if the problem is corrected. Again sorry for delay I was caught up with other work. © kashinO8 says 07/10/2014 at 8:56 AM Sorry that it’s just a copy paste error, it should be 192.168.2.12. Talso try your suggestion to refresh iptables, but the problem still exist: external PC can’t ping VM instance via floating IP & VM instance can’t ping external PC. © akilesh1597 says 07/10/2014 at 12:23 PM ps:tosskb wordpress. comn/20"406 1!managing-openslack:-nieralataexteral-network-ir-on-inerface! s986ss12015 Managing Operstack InlecraiDatafExtecral network n ane interface | Free and Open Source Software Knowledge Base can you execute the same command (ip netns exec qrouter- iptables -t nat -nvL) and post it again. © kashin08 says: 09/10/2014 at 6:26 AM Sure, please refer to the link below: © akilesh1597 says: 21/10/2014 at 7:15 PM Hi, Sorry, wasn’t able to spot any problem with iptable rules. If all interfaces(including the virtual ones you create manually) are in promiscous mode, security group rules allow ingress access for the protocol(icmp/ssh) you are using, then it should work. Ihave tested the methodology described in this post and it does work. May be your setup has some problem outside of openstack. 16/09/2014 at Hi Akilesh, Nice post. Thave a single physical server where DevStack is installed. It serves as the controller, network and compute node. The machine has one NIC, but I need to create 2 networks — one for external and another for management. So I created a virtual NIC — eth0:1 with 192.168.0.99 as the IP address. The host IP is 172.26.1.74. Then created 2 bridges as follows : sudo ovs-vsctl add-br br-mng sudo ovs-vsetl add-br br-ext sudo ovs-vsctl add-port br-mng eth0:1 sudo ovs-vsctl add-port br-ext eth0 Talso created 2 networks namely - external and management. The issue is when I create a VM instance on the management network, it is not reachable(unable to ping). Following is my /etc/network/interface details: auto lo iface lo inet loopback auto eth0 face eth0 inet manual up ip address add 0.0 dev $IFACE up ip link set $IFACE up down ip link set $IFACE down auto br-ext iface br-ext inet static address 172.26.1.74 ps:tosskb wordpress. comn/20"406 1!managing-openslack:-nieralataexteral-network-ir-on-inerface! 2086xsva0s Managirg Operetck eral atatxteral network in oe irerface| Free and Open Source Sofware Knowledge Base netmask 255,.255.255.0 network 172.26.1.0 broadcast 172.26.1.255 gateway 172.26.1.1 # dns-* options are implemented by the resolvconf package, if installed dns-nameservers 172.26.11.156 172.21.133.10 dns-search oss.vyatta.net auto eth0:1 iface eth0:1 inet manual up ip address add 0.0 dev $IFACE up ip link set $IFACE up down ip link set $IFACE down auto br-mng face br-mng inet static address 192.168.0.99 netmask 255.255.0.0 network 192.168.0.0 broadcast 192.168.255.255 Reply © akilesh1597 says: 23/09/2014 at 1:53 PM How did you create a virtual nic? Reply 8. akilesh1597 says: 15/09/2014 at 5:25 PM es related to floatingip, routing, nating please check 9. Andrew Mann say: 10/09/2014 at 4:01 AM For a more secure variant, apply VLAN tagging to your br-eth0 and br-etht patch port, and segregate your management and external networks. in one step: ovs-vsctl — add-port br-eth0 ptch-eth0-eth1 tag=1 — set interface ptch-eth0-eth1 type=patch optio ptch-eth1-eth0 — add-port br-eth1 ptch-eth1-eth0 — set interface ptch-eth1-eth0 type=patch options:peer=ptch-eth0-eth1 se ovs-vsctl to make and link these ports This makes the veth patch port an access port with vlan=1, so traffic dumped onto “br-eth1” always ends up on vian1 on your physical network. You can do the same to make a bridge for “eth2' that always ends up on vlan2. Enabled vlan 1 and 2 on your switch (in addition to 100- 200 for your tenant networks). Now you still have the shared 1gbit link, but you don’t have the security concerns of leaky traffic between networks. ps:tosskb wordpress. comn/20"406 1!managing-openslack:-nieralataexteral-network-ir-on-inerface! 2186xsva0s Managirg Operetck eral atatxteral network in oe irerface| Free and Open Source Sofware Knowledge Base Reply 0. Davincho sa 08/09/2014 at 3:28 PM Another question: When I configure eth0 to be part of ex-br and I assign the bridge the former IP of the port like so: ovs-vsctl add-port br-ex eth ifconfig br-ex 128.131.168.15 after y starting the network service “service network restart” the bridge loos use CentOS and tried to create a file in /etc/sysconfig/network-scripts/ifcfg-br-ex and define an IP address in it, but it gets ignored. Additionally when I create a bridge manually without using ovs-vsctl, can it still be used by OpenVSwitch? Thanks Reply © Johnson D says: 09/09/2014 at 5:39 AM Can you paste the /etc/sysconfig/network-scripts/ifcfg-br-ex file here? Also can you paste the command you used to create bridge: Reply © Davincho says: 09/09/2014 at 5:39 PM DEVICE=br-ex TYPE=Bridge ONBOOT-=yes BOOTPROTO-static IPADDR=128.131.168.152 NETMASK=255.255.255.0 GATEWAY=128.131.168.100 and for ethO DEVICE_INFO_HERE ONBOOT=y BOOTPROTO=none PROMISC=yes BRIDGE=br-ex Although with this config, I would get an error when I restart the network service as the system tells my that the bridge already e The bridge was created with ovs-vsctl add-br br-ex 09/09/2014 at 7:09 PM ps:tosskb wordpress. comn/20"406 1!managing-openslack:-nieralataexteral-network-ir-on-inerface! zaness12015 Managirg Operetck eral atatxteral network in oe irerface| Free and Open Source Sofware Knowledge Base Idid not have an immediate answer, but I googled this for you. ipermail/discuss/2011-October/005845,html. I think this should © Davincho says: 09/09/2014 at 7:24 PM Thanks for the link. But in general how would you accomplish that the configuration of OpenVSwitch bridges stays the same without a script after restarting the network service? akilesh1597 say: 10/09/2014 at 8:50 AM The openvswitch bridges, their ports and their configuration are maintained by ovsdb. Assigning IP address to bridges instead of assigning to the port is more of a hack. Openvswitch does not keep track of it. Making use of network-scripts is the only option left. A more crude way is to create a shell script and invoke it on bootup. © Davincho says: 10/09/2014 at 12:45 PM Okay so if I follow the steps from the official documentation: yum/content/neutron-m12- sctl add-br br-ex ict add-port br-ex ethO This would result into a connection lost, as eth0 is not reachable any more. According to p=openvswitch:a=blob_plain:f=FAQ! hb=-HEAD “A physical Ethernet device that is part of an Open vSwitch bridge should not have an IP addre: then that IP address will not be fully functional.” which means that “You can restore functionality by moving the IP address to an OpenvSwitch “internal” device, such as the network device named after the bridge itself”. So essentially the documentation of OpenvSwitch proposes a hack of assigning an IP address to a bridge? What is the official way of configuring OpenStack then, I am kind of confused ... Cheers © akilesh1597 says: 10/09/2014 at 1:36 PM It is a hack. The proper way to do is to use eth0 for internal network and other interfaces(ethI, eth2 etc) to add to br-eth1(data network) and br-ex(external network). eth0 is not added to any bridge and so is available for the host and can have a valid ip address assigned. 1. Davincho says: ps:tosskb wordpress. comn/20"406 1!managing-openslack:-nieralataexteral-network-ir-on-inerface! 26xsva0s Managirg Operetck eral atatxteral network in oe irerface| Free and Open Source Sofware Knowledge Base 08/09/2014 at 1:00 PM Thanks for this useful blog entry! Ithink you made a mistake for the last configuration scenario, instead of: #Attach bridges using veth pair ovs-vsetl add-port br-eth eth1-br-proxy ovs-vsctl add-port br-ex ex-br-proxy ovs-vsctl add-port br-eth0 proxy-br-eth1 ovs-vsctl add-port br-eth0 proxy-br-ex It should be: ovs-vsctl add-port br-eth eth1-br-proxy ovs-vsctl add-port br-ex ex-br-proxy ovs-vsctl add-port br-proxy proxy-br-eth1 ovs-vsctl add-port br-proxy proxy-br-ex Cheers Reply © akilesh1597 say: 09/09/2014 at 6:57 AM Thank you for reviewing. I have updated now. Reply 2. Amit says: 05/09/2014 at 8:19 PM Hi guys could use some help here if you have time! Basic: I have 3 NUCs where I am installing openstack on. One is controller, one is network and the other compute. The NUCs only have one interface each wherein my problem lies! What I have done is create VLANs for each interface on the NUCS. So for instance on the network host I have 3 VLANs. One for management, one for instance tunnels and one is the “unnumbered” interface for the br-ex. So pretty much everything works great EXCEPT that I cant get the br-ex to work (so cant ping the public IP or get to internet). I can however bring up an instance and get it working on the tenant network. I have no clue where I am going wrong and can send you my configs if you got some time to look at them I would GREATLY appreciate it. Reply © akilesh1597 says: 09/09/2014 at 7:06 AM You are not alone with this problem. Several people updated similar issues. I am planning ona post solely for the purpose of debugging such issues. Reply © Amit says: 11/09/2014 at 12:56 AM Thank you that would be great as I am at a stopping point and cant go any further. Please let us know when it is up. Thanks! © akilesh1597 says: ps:tosskb wordpress. comn/20"406 1!managing-openslack:-nieralataexteral-network-ir-on-inerface! 266xsva0s Managirg Operetck eral atatxteral network in oe irerface| Free and Open Source Sofware Knowledge Base 15/09/2014 at 5:22 PM Please check this https: [-agent/. I have not updated the debugging steps yet but may be it would help you. Reply 3. Viswesuwara nathan says: 04/09/2014 at 7:14 AM Tam trying to configure openstack in two machines and I need you guide in setting up multiple interface on single nic; Refer the image in the link to understand the network topology https://www.dropbox.com/s/n5d5blv4a26rt3c/OpenStack_Network.bmp?dl=0 that I have configured. Tam facing issue in bringing up additional network (Say Mgmt or data) in the same interface (etho) which was initially configured for external network. Now, I was able to ping from host machine A and B using external network but if I configure br-eth1 to 10.20.0.15 in host A and 10.20.0.16 in host B; I was not able to ping them I tried running ‘tcpdump -i eth0 -v icmp’ in hostA while pining host B (10.20.0.16) and I could not see any packet in eth0; Is I configured something wrong? Please guide me on this. Thanks 04/09/2014 at 1:31 PM Please check the image and commands towards the end of the blog Reply 4, kashin08 say 29/08/2014 at 2:20 AM First of all, thanks for your detail setup I followed your steps to install Icdhouse in single node with single NIC environment. After all, i can launch instances and it works like a charm. But the only problem is “VM instance can’t access external network”. The network topology can refer to this image (https://www.dropbox.com/s/c66n7j3wg1 miaog/network-topology .png?dl=0). M1 and VM2 can ping each other successfully but can't ping outside world even the external gateway (192.168.1.1). VM3 is attached to external network directly, but it still can’t access outside world either (even fail to ping 192.168.1.1). Could you help to resolve this issue? and which information should I provide to help to dig into this? Reply © akilesh’ 597 says: ps:tosskb wordpress. comn/20"406 1!managing-openslack:-nieralataexteral-network-ir-on-inerface! 256ss12015 Managirg Operetck eral atatxteral network in oe irerface| Free and Open Source Sofware Knowledge Base 08/2014 at 9:33 AM 1. Can your instances ping 192.168.2.1 2. Your VMS technically should not be reachable from external network. Instead you should assign floating ip to the VMS from the external network. Your VMS should be reachable only using that floating ip. 3. VMB is not reachable from external network. I am not sure if this is valid. Because the external network serves the purpose of bringing the vm traffic out of openstack. I have not tried launching vms in it. Let me try and get back. Meanwhile you try if 1 and 2 work for you. Reply © kashin08 says: 01/09/2014 at 1:15 AM The result of action 1 & 2 are listed below: 1. VMI & VM2 can ping 192.168.2.1 YM1 & VM2 can ping 192.168.1.100 YM1 & VM2 can’t ping 192.168.1.1 MB can't ping 192.168.2.1 ‘VM can’t ping 192.168.1100 ‘VMB can’t ping 192.168.1.1 2. Lassociate a floating IP (192.168.1.101) for VM1, but i still can’t reach VMI from external network. akilesh1597 says: 04/09/2014 at 9:45 AM Please do check if eth0 is set in promiscous mode. Check if your external gateway can ping 192.168.1.100. If no Then we have to use tcpdump to check where exactly the packet is getting dropped. Start a continuous ping from the external gateway to your instance’s floating ip. Then issue tcpdump on each of the below interfaces and tell me on which interfaces you see the icmp echo request message and on which you dont, 1.eth0 2. The interface corresponding to the port connecting openstack router to the external s inside the routers namespace. To execute tepdump inside the namespace you have to use ‘ip netns exec tepdump -lennvi 3. phy-br-ex 4. int-br-ex Tknow its tough to follow. specially if you are a beginner. But you have to help me to help network. This interface will exis you. 09/09/2014 at 3:09 AM tepdump result Case 1: Ping 192.168.1.1 from VM [taprxxx] & [qbroxx] & [qvbxxox] & [qvorre,] 22:24:16.462247 IP 192.168.2.10 > 192.168.1.1: ICMP echo request, id 28417, seq 0, length ps:tosskb wordpress. comn/20"406 1!managing-openslack:-nieralataexteral-network-ir-on-inerface! a6xsva0s Managirg Operetck eral atatxteral network in oe irerface| Free and Open Source Sofware Knowledge Base 64. 22:24:17 464266 IP 192.168.2.10 > 192.168.1.1: ICMP echo request, id 28417, seq 1, length 64 22:24:18.464738 IP 192.168.2.10 > 192.168.1.1: ICMP echo request, id 28417, seq 2, length 64 22:24:19.461006 IP 192.168.1.101 > 192.168.2.10: ICMP host 192.168.1.1 unreachable, length 92 22:24:19.461302 IP 192.168.1.101 > 192.168.2.10: ICMP host 192.168.1.1 unreachable, length 92 22:24:19.461313 IP 192.168.1.101 > 192.168.2.10: ICMP host 192.168.1.1 unreachable, length 92 [br-int] (no any packages related to 192.168.1.101) Case 2: Ping 192.168.1.101 (VM floating IP) from PC [br-eth0] 2 594347 ARP, Request who-has 192.168.1.101 tell 192.168.1.31, length 28 592029 ARP, Request who-has 192.168.1.101 tell 192.168.1.31, length 28 22:17:18.610343 ARP, Request who-has 192.168.1.101 tell 192.168.1.31, length 28 (no arp reply) [proxy-br-ex] (no any packages related to 192.168.1.101) [proxy-br-eth1] (no any packages related to 192.168.1.101) And here is my “ovs-vsctl show” result: Bridge br-int fail_mode: secure Port int-br-ex Interface int-br-ex Port br-int Interface br-int type: internal Port “qvoa9c3c3c0-9b” tag: 1 Interface “qvoa9c3c3c0-9b” Port “qr-2adafd77-8b” tag: 1 Interface “qr-2adafd77-8b” type: internal Port “tapb9df269f-ec” ce “tapb9df269F-ec” type: internal Port “int-br-ethi” Interface “int-br-eth1” ps:tosskb wordpress. comn/20"406 1!managing-openslack:-nieralataexteral-network-ir-on-inerface! 2188xsva0s Managirg Operetck eral atatxteral network in oe irerface| Free and Open Source Sofware Knowledge Base Bridge “br-eth0” Port “br-eth0” Interface “br-eth0" type: internal Port “eth0" Interface “eth0" Port “phy-br-eth0” Interface “phy-br-eth0" Port proxy-br-ex Interface proxy-br-ex Port “proxy-br-eth1” Interface “proxy-br-eth1” Bridge br-ex Port br-ex Interface br-ex type: internal Port “qg-2648325b-6¢” Interface “qg-2648325b-6e” type: internal Port phy-br-ex Interface phy-br-ex Port ex-br-proxy Interface ex-br-proxy Bridge “br-eth1” Port “phy-br-eth1” Interface “phy-br-eth1” Port “br-ethi” Interface “br-eth1" typ Port “eth1-br-proxy” Interface “eth1-br-proxy” ovs_version: “2.0.1” internal 5. Jagadeesh sa 20/08/2014 at 4:09 PM Hi Akilesh, We are working on openstack havana (ubuntu 12.04 LTS) 2 node setup. Where 1 node is controller + network node and other node is compute node. Both nodes are having 1 NIC. We are using GRE tunnelling mechanism as our network plugin. I’m facing issue while launching the Virtual instance from controller. Error: “Connection to Neutron failed: Maximum attempts reached”, Seems to be a networking issue. Could you please help us in this regard. Please give us an idea how should we configure the network. Regrads Jagadeesh Reply ps:tosskb wordpress. comn/20"406 1!managing-openslack:-nieralataexteral-network-ir-on-inerface! 26xsva0s Managirg Operetck eral atatxteral network in oe irerface| Free and Open Source Sofware Knowledge Base © akilesh1597 say: 21/08/2014 at 7:27 AM This is not a neutron sue. The error occurs when nova fails to contact neutron server for vif creation prior to instance creation. Issue ‘keystone catalog’ command on controller node and check if the url for the neutron endpoint is reachable from controller. You have to follow the exact instructions given under section ‘Using the same Interface for all Networks’, except that the compute node need not create ‘br-ex’, Reply © Jagadeesh says: 21/08/2014 at 4:14 PM Hey Akiles Thanks for your quick response. Regarding instructions under section ‘Using the Same Interface for all Networks’, they seems to be for VLAN OVS plug-in. I'm using GRE OVS plug-in, is the instructions are same in my case also ? If not, please provide me the instructions. The reason I’m asking I haven't configured ‘network_vlan_ranges’, ‘bridge_mapping’ etc in conf files. I just followed openstack havana installation guide and just made/enable settings relevant to GRE plug-in only. Please clarify. Also, below is content in n/w interface file of my controllernetwork node (/etc/network/interface). Note that i‘m running openstack on ubuntu 12.04 LTS. auto lo iface lo inet loopback auth ethO face eth0 inet manual up ifconfig SIFACE 0.0.0.0 up up ifconfig SIFACE promise on down ip link set $IFACE promise off down ifconfig SIFACE down auto br-ex iface br-ex inet static address 192.168.1.53 netmask 255.255.255.0 gateway 192.168.1.1 The same(above) is for compute node also (just ip address is different) As per installation guide i’ve created 2 bridges br-ex and br-int on both nodes. br-tun is internally created by plug-in. I’ve added eth0 as port to br-ex bridge on both nodes. T’ve executed ‘keystone catalog’ command and I can see end-point of neutron (servic network) as “publicURL http:/controller:9696”. I think it is proper. Also in the previous comment you mentioned to check the url is reachable from controller, how to do that ? ps:tosskb wordpress. comn/20"406 1!managing-openslack:-nieralataexteral-network-ir-on-inerface! 2086xsva0s Managirg Operetck eral atatxteral network in oe irerface| Free and Open Source Sofware Knowledge Base T’'m struck !!! no clue what to do, that’s why i've provide the information which I think might give you more idea about my case. Please guide me in this regard. Appreciate your response. Thanks. © akilesh1597 say: 22/08/2014 at 6:18 AM The only difference is that you need not create br-eth, no need for defining bridge_mappings. You only have to define local_ip in mI2_conf.ini, which I believe you have done. I missed to read that you are using gre mode, but still what you have done is already correct. As I said earlier, this error occurs when nova can not reach neutron. your keystone catalog has returned http://controller:9696/. Make sure the name controller is resolvable by all your nodes. Or add an entry in /etc/hosts. By reachable I mean you should be able to ping controller, your neutron-server is running(test using service neutron-server status) and also it is listening on port 9696(test using netstat -tulnp) Re © Jagadeesh say 22/08/2014 at 4:06 PM Since we are using GRE plugin .. have concern regarding the commnds you have mention above, ip link add proxy-br-eth1 type veth peer name eth1-br-proxy ip link add proxy-br-ex type veth peer name ex-br-proxy Thelieve veth is for VLAN plug-in. If possible, pleaes provide me the set of command which i should run with GRE plug-in like, what bridges i’ve to create and the ports/proxy which i’ve to attach. © akilesh1597 says: 30/08/2014 at 9:35 AM Twill upload the an command and a supporting image some time on 1st of september. check them. © akilesh1597 says: 04/09/2014 at 1:32 PM Please check the setup at the end of the blog. It is exactly your requirement. Reply 6. Ehwan Kho says: 12/08/2014 at 3:43 AM Hil I need some confirmation does data network need to have an IP address or not? as I could not get it. Yours is just a single NICs which ‘I think would’ not matter of putting the ip. Or do T need to put also ip for br-ex? I know it is on multiple NICs which probably solved as you are now dealing with single NIC. Hope you can show up your configuration if possible. Ehwan Unlucky neutron user Reply © akile 1597 says: ps:tosskb wordpress. comn/20"406 1!managing-openslack:-nieralataexteral-network-ir-on-inerface! a6xsva0s Managirg Operetck eral atatxteral network in oe irerface| Free and Open Source Sofware Knowledge Base 12/08/2014 at 4:11 AM Rephrasing your query as does the interface/bridge connecting to data network or external network need an ip? No it doesn’t. You just have to set the interface up and in promiscuous mode. The interface connecting to internal network alone needs an ip, because that will alone be used by host. The others are used by openstack for laying the overlay network. Hope its clear. Reply © akilesh1597 says: 12/08/2014 at 4:13 AM When using open source products you should not consider yourselves unlucky. Its a privilege to share the burden and success of the community. Reply 7. Lokesh Jawane says: 10/08/2014 at 8:17 PM HI, Thave single m/c with single NIC & i want to assign IP to openstack VM from my LAN via(dhcp/static) not therought the insternal DHCP. Do you have any idea hot to disable internal DHCP & assign IP of LAN network. Reply © akilesh1597 say: 11/08/2014 at 9:34 AM Having a single or multi nic is not an issue here. You can use my info to install a working openstack in single machine. After this you have to choose one of the following 1. Create all your instances in external network directly, instead of creating it in a private network. 2. configure dnsmasq(in whichever machine hosting neutron-dhep-agent) to run as relay agent instead as server mode. Doing so is beyond openstack. You should modify the source code of neutron-dhep-agent(this is the service that starts dnsmasq processes, that serve as dhep servers) to do this. Reply 8, Tahder say: 08/08/2014 at 4:05 AM Your article give me an idea of my problem which is an awesome article. But Ihave query regarding with 2 NICs, I know this article using one as I confused and need you're insights. Do Ineed not to run the ip link add ...? Based on my openstack, the vms can get IP using flat network, but using ml2/vlan could not reach, it seems dhep could not get through vms. I think the problem is somewhere here. Reply © akilesh1597 says: 08/08/2014 at AM ps:tosskb wordpress. comn/20"406 1!managing-openslack:-nieralataexteral-network-ir-on-inerface! 36xsva0s Managirg Operetck eral atatxteral network in oe irerface| Free and Open Source Sofware Knowledge Base This article is for people who want to install openstack on single machine with single nic. There are many ways in which a person can configure openstack. You have to give me more information regarding your set up. are you doing single node or multi node? If you have 2 nics you could use one nic for both external and internal network use another for data network. Asumin you have single machine with 2 nics I would do ovs-vsetl add-br br-ex ctl add-br br-eth1 ‘ctl add-port br-eth1 eth1 #this is for data network ict] add-port br-ex eth0 # this is for external network. but now eth0 can not be used by host. so give ifconfig br-ex up # eth0’s ip address to br-ex. Now both external and internal network use etho No need of creating veth pairs. Then do ‘ip link set up promisc on’ on all interfaces involved. ov: ov: ov: Reply © Tahder say: 08/08/2014 at 10:49 AM Yes, 1am doing a multi-nodes (controller, network and 1 compute node). I will give this a try that is what I done except for adding the ‘ip link set up promisc on’ based on my assumption it was only br-ex will be in promise as in the documentations... Your article jing link. lol, thanks a lot for sharing your knowledge. It could save lot of NICs. Using this one interface do I need to enable the net.ipv4.ip_forward=1 in sysetl.conf? Just verification with the “ifconfig br-ex up” as you said eth0’s ip address to br-ex, meaning we have to remove the eth0 address? as mine set on static ip 192.168.1.100 or do we need to have a new IP for example? which also in contrasting to one of your reply to Ehwan that states “external network need an ip? No it doesn’t”. Now confused. © akilesh1597 say: 08/08/2014 at 11:45 You have to do ‘net.ipv4.ip_forward=1' only on the network node, because that is where your neutron-I3-agent will be running and that is where virtual routers will exists. On the controller you do not need any bridges, just eth0 with normal configuration is enough. The compute node requires just two interfaces, one for data network(you add eth! to br-eth1) and another for internal network (use eth0, no need to add any bridges other than br-eth1 and br-int). Only on the network node you would need three interfaces eth0 to use for internal network, eth] to add to br-eth] and use for data network, and eth2 to add to br-ex and use for external network. If you combine eth0 and eth? functionality as I said in my previous reply you should be fine. Reply © akilesh1597 say: ps:tosskb wordpress. comn/20"406 1!managing-openslack:-nieralataexteral-network-ir-on-inerface! saxsva0s Managirg Operetck eral atatxteral network in oe irerface| Free and Open Source Sofware Knowledge Base 3108/2014 at 4:51 AM If your external, data and internal network do not share the same NIC, only the NIC connecting to internal network need have the IP address. my suggestion to you was to share eth0 between external and internal network which is why I asked you to remove eth0s ip address, add eth0 to br-ex and finally assign eth0’s ip address to br-ex. Lam planning to upload an Image to my post that would make things clear. So hang on if you do not understand. Reply © Tahder says: 14/08/2014 at 4:14 AM The image gave me a clear view of what really you're writing and it was very helpful to understand. Yes I got what you mean, thanks for that information. Is it possible to make a configuration file on the network? so it will not run the ovs-vsctl. As some article mentioned to put the ovs as the device type, example for fedora to put in br-ex DEVICE=eth2 DEVICETYPE-ovs TYPE=OVSPort ONBOOT-=yes OVSBOOTPROTO=none OVS_BRIDGE-br-eth2 9. Samit Jain says 01/08/2014 at 4:38 AM Actually azure provides only one NIC. It does not provi networking guy. Can you help me, how to resolve the i je more than one NIC. Iam not a sue, ilesh1597 says 01/08/2014 at 6:35 AM Whaat is the distro of your host? Reply © Samit Jain says 08/08/2014 at 8:20 AM itis ubuntu 12.04 © akilesh1597 says: 08/08/2014 at 10:08 AM Can you post your /etc/network/interfaces. Reply © akilesh1597 say: 08/08/2014 at 10:15 AM Skip the below two steps from my post ovs-vsctl add-port br-eth0 eth ifconfig br-eth0 up ps:tosskb wordpress. comn/20"4061managing-openslack:-niernaléaiaexteral-network-ir-on-inerfaca! 96xsva0s Managirg Operetck eral atatxteral network in oe irerface| Free and Open Source Sofware Knowledge Base Do the rest. Edit your /etc/network/interfaces file to look like below and restart your instance. That should add eth0 to br-eth0 and then set the dhep assigned ip address to br-eth0. Come back if anything goes wrong. auto eth0 face eth0 inet manual auto br-eth0 iface br-eth0 inet dhep bridge_ports ethO Reply © Samit Jain says 08/08/2014 at 10:21 AM Thanks for your prompt response. I will give it a try. 0. Samit Jain says: 30/07/2014 at 5:12 AM Tam trying to install openstack in single node on ubuntu 12.04 VM on azure. It has single NIC so I followed above article to set bridges etc. But after that networking of my vm breaks and I have to shutdown vm. I tried 4 times and every time the result is same. Please suggest. Reply © akilesh1597 say: 07/2014 at 4:20 PM Those instructions work when you have direct keyboard access to the host, not when the host is remote and certainly not when host itself is virtualized. If your host is virtual then why not add more NICs. It is possible in aws and also in openstack. I am not sure of azure though. Reply ‘1. Deepak C Shetty says: 16/06/2014 at 11:07 AM ‘Yup, know what u only need to know Reply 12. akilesh1597 says: 16/06/2014 at 10:55 AM You are correct. What I meant is that there is going to be a significant reduction in the necessity to know the networking stuff. Only those in core business need to know. Which is good anyway. We can concentrate on things of our own interest as you say. Reply 13. Deepak C Shetty says 16/06/2014 at 9:35 A. ps:tosskb wordpress. comn/20"406 1!managing-openslack:-nieralataexteral-network-ir-on-inerface! 346sis12015 Managing Opestack hieralDataExteral retwerkin oe interface | Free ar Open Source Sofware Knowledge Base Actually, thats not entirely true and thats not the right perspective to look at cloud. IMHO, cloud enables people who don’t and shouldn't need to know abt networking from a business, end-user perspective. network internals will still be a sought after area but by those people who want to make a career out of it and who have the interest/passion.. bcos for neutron to work and to implement it.. n/wing knowledge is a must. So in short, cloud helps those who want to use n/wing but don’t want to know the internals of it My 2 cents 4. 14/06/2014 at 2:23 PM Thanks for the nice article... hoping to read more abt networking basics that are pre-req to understand and use neutron for a non-networking guy likeme - deepak Reply © akilesh1597 says: 15/06/2014 at 8:02 AM Neutron is for non-networking guys actually. One bad thing about cloud is that it will slowly root out the necessity of knowledge in the area of basic network and server administration. Anyways, I'll think about a post on linux bridging and networking in general. Reply 5. Mike Spreitzer say’ 11/06/2014 at 3:21 PM Should ovs-vsctl add-port br-ex eth1-br-proxy be ovs-vsctl add-port br-ex ex-br-proxy 2 Reply © akilesh1597 say: 12/06/2014 at 12:27 PM You are correct. I have updated the same. Thank you. Reply Create a free website or blog at WordPress.com. ~ The Syntax Theme. @ Follow Follow “Free and Open Source Software Knowledge Base” ps:tosskb wordpress. comn/20"406 1!managing-openslack:-nieralataexteral-network-ir-on-inerface! 96ss12015 Managing Operstack IneralDatalExterral network in ane interface | Free and Open Source Software Knowledge Base Build a website with WordPress.com ps:tosskb wordpress. comn/20"406 1!managing-openslack:-nieralataexteral-network-ir-on-inerface!