ASM rebalancing commands

For ASM rebalancing good commands always come handy. This one shows current ASM disks with group info and mount status with very well formed way.

 column path format a20  
 set lines 132  
 set pages 50  
 select path, group_number group_#, disk_number disk_#, mount_status,  
 header_status, state, total_mb, free_mb  
 from v$asm_disk   
 order by group_number;  

The next one does good formatting for showing disks in failgroup and diskgroup

 SET ECHO    OFF  
 SET FEEDBACK  6  
 SET HEADING   ON  
 SET LINESIZE  180  
 SET PAGESIZE  50000  
 SET TERMOUT   ON  
 SET TIMING   OFF  
 SET TRIMOUT   ON  
 SET TRIMSPOOL  ON  
 SET VERIFY   OFF  
 CLEAR COLUMNS  
 CLEAR BREAKS  
 CLEAR COMPUTES  
 COLUMN disk_group_name    FORMAT a25      HEAD 'Disk Group Name'  
 COLUMN disk_file_path     FORMAT a20      HEAD 'Path'  
 COLUMN disk_file_name     FORMAT a20      HEAD 'File Name'  
 COLUMN disk_file_fail_group  FORMAT a20      HEAD 'Fail Group'  
 SELECT  
   NVL(a.name, '[CANDIDATE]')            disk_group_name  
  , b.path                      disk_file_path  
  , b.name                      disk_file_name  
  , b.failgroup                   disk_file_fail_group  
 FROM  
   v$asm_diskgroup a RIGHT OUTER JOIN v$asm_disk b USING (group_number)  
 ORDER BY  
   a.name  
 /  

Adding and removing normal disks from diskgroup

 alter diskgroup DISKDG add disk 'NEWDISK' rebalance power 11;  
 alter diskgroup DISKDG drop disk 'OLDDISK' rebalance power 11;  

Adding and removing several disks from diskgroup

 alter diskgroup DISKDG add disk 'ORCL:NEWDISK', 'ORCL:NEWDISK2' rebalance power 11;  
 alter diskgroup DISKDG drop disk 'OLDDISK', 'OLDIDSK2' rebalance power 11;  

Adding disks into failgroup

 ALTER DISKGROUP FAILDISKDG ADD   
 FAILGROUP FAILONE DISK 'ORCL:NEWDISK3', 'ORCL:NEWDISK4'  
 FAILGROUP FAILTWO DISK 'ORCL:NEWDISK5', 'ORCL:NEWDISK6'  
 REBALANCE POWER 11;  

Checking status of ASM rebalance (remember to check ASM alert log, this will only give estimate of rebalancing)

 SELECT group_number, operation, state, power, est_minutes FROM v$asm_operation;  

Create new ASM diskgroup for voting disk

1) Create new diskgroup for ASM diskgroup

CREATE DISKGROUP VOTEVPLEX EXTERNAL REDUNDANCY DISK 'ORCL:VOTEVPLEX' ATTRIBUTE 'au_size'='4M';

alter diskgroup VOTEVPLEX set attribute 'compatible.asm'='11.2;

2) Migrate voting file to new ASM diskgroup

[root@rac1 ~]# su - oragrid

+ASM1_rac1 $ sqlplus / as sysasm

SQL> CREATE DISKGROUP VOTEVPLEX EXTERNAL REDUNDANCY DISK 'ORCL:VOTEVPLEX' ATTRIBUTE 'au_size'='4M';

SQL> alter diskgroup VOTEVPLEX set attribute 'compatible.asm'='11.2';

Diskgroup altered.


Diskgroup created.

+ASM1_rac1 $ ocrcheck
Status of Oracle Cluster Registry is as follows :
         Version                  :          3
         Total space (kbytes)     :     262120
         Used space (kbytes)      :       4384
         Available space (kbytes) :     257736
         ID                       :   70121363
         Device/File Name         :     +OCRDG
                                    Device/File integrity check succeeded

                                    Device/File not configured

                                    Device/File not configured

                                    Device/File not configured

                                    Device/File not configured

         Cluster registry integrity check succeeded

         Logical corruption check bypassed due to non-privileged user


+ASM1_rac1 $ crsctl   query css votedisk
##  STATE    File Universal Id                File Name Disk group
--  -----    -----------------                --------- ---------
 1. ONLINE   12e224acaa594fcfbf245fd6486e1ed0 (ORCL:VOTEDC1) [VOTEDG]
 2. ONLINE   355415a275c64fabbf0159a43d7bac62 (ORCL:VOTEDC2) [VOTEDG]
Located 2 voting disk(s).

3) Check results

+ASM1_rac1 $ crsctl  replace votedisk +VOTEVPLEX
Successful addition of voting disk 0d588bf13f684f32bfb3d45688c28826.
Successful deletion of voting disk 12e224acaa594fcfbf245fd6486e1ed0.
Successful deletion of voting disk 355415a275c64fabbf0159a43d7bac62.
Successfully replaced voting disk group with +VOTEVPLEX.
CRS-4266: Voting file(s) successfully replaced

+ASM1_rac1 $ crsctl   query css votedisk
##  STATE    File Universal Id                File Name Disk group
--  -----    -----------------                --------- ---------
 1. ONLINE   0d588bf13f684f32bfb3d45688c28826 (ORCL:VOTEVPLEX) [VOTEVPLEX]

How to configure Multipath disks for Oracle ASM use Step-By-Step

0) Scan your disks

for i in 0 1 2 3 4 5 6; do echo "- - -" > /sys/class/scsi_host/host$i/scan; done

0,5) Find your disks

[root@rac1 ~]# multipath -ll

1) On both nodes, change oracleasm configuration file 

[root@rac1 ~]# nano -w /etc/sysconfig/oracleasm
[root@rac2 ~]# nano -w /etc/sysconfig/oracleasm

Add mpath

# ORACLEASM_SCANORDER: Matching patterns to order disk scanning
ORACLEASM_SCANORDER="mpath dm"

2) Format disk and check output on both nodes

[root@rac1 ~]# fdisk /dev/mapper/mpathjl

Device contains neither a valid DOS partition table, nor Sun, SGI or OSF disklabel

Building a new DOS disklabel with disk identifier 0x35dd781b.

Changes will remain in memory only, until you decide to write them.

After that, of course, the previous content won't be recoverable.

Warning: invalid flag 0x0000 of partition table 4 will be corrected by w(rite)

WARNING: DOS-compatible mode is deprecated. It's strongly recommended to

         switch off the mode (command 'c') and change display units to

         sectors (command 'u').

Command (m for help): p

Disk /dev/mapper/mpathjl: 1074 MB, 1074462720 bytes

255 heads, 63 sectors/track, 130 cylinders

Units = cylinders of 16065 * 512 = 8225280 bytes

Sector size (logical/physical): 512 bytes / 512 bytes

I/O size (minimum/optimal): 512 bytes / 512 bytes

Disk identifier: 0x35dd781b

              Device Boot      Start         End      Blocks   Id  System

Command (m for help): n

Command action

   e   extended

   p   primary partition (1-4)

p

Partition number (1-4): 1

First cylinder (1-130, default 1):

Using default value 1

Last cylinder, +cylinders or +size{K,M,G} (1-130, default 130):

Using default value 130

Command (m for help): w

The partition table has been altered!

Calling ioctl() to re-read partition table.

WARNING: Re-reading the partition table failed with error 22: Invalid argument.

The kernel still uses the old table. The new table will be used at

the next reboot or after you run partprobe(8) or kpartx(8)

Syncing disks.

[root@rac1 ~]# fdisk -l /dev/mapper/mpathjl

Disk /dev/mapper/mpathjl: 1074 MB, 1074462720 bytes

255 heads, 63 sectors/track, 130 cylinders

Units = cylinders of 16065 * 512 = 8225280 bytes

Sector size (logical/physical): 512 bytes / 512 bytes

I/O size (minimum/optimal): 512 bytes / 512 bytes

Disk identifier: 0x35dd781b

              Device Boot      Start         End      Blocks   Id  System

/dev/mapper/mpathjlp1               1         130     1044193+  83  Linux

[root@rac2 ~]# fdisk -l /dev/mapper/mpathjl

Disk /dev/mapper/mpathjl: 1074 MB, 1074462720 bytes

255 heads, 63 sectors/track, 130 cylinders

Units = cylinders of 16065 * 512 = 8225280 bytes

Sector size (logical/physical): 512 bytes / 512 bytes

I/O size (minimum/optimal): 512 bytes / 512 bytes

Disk identifier: 0x35dd781b

              Device Boot      Start         End      Blocks   Id  System

/dev/mapper/mpathjlp1               1         130     1044193+  83  Linux

2.1) Copy fdisk partition

sfdisk -d mpathjl > /tmp/50GB_LUN.txt

2.2) Apply fdisk partition to empty LUN

sfdisk mpath < /tmp/50GB_LUN.txt

for i in $(cat /tmp/100gb_mpath); do sfdisk $i < /tmp/mpath100gb.txt done

3) Change ownership of mpath disk

[root@rac1 ~]# fdisk -l "/dev/mapper/mpathjl"

Disk /dev/mapper/mpathjl: 1074 MB, 1074462720 bytes

255 heads, 63 sectors/track, 130 cylinders

Units = cylinders of 16065 * 512 = 8225280 bytes

Sector size (logical/physical): 512 bytes / 512 bytes

I/O size (minimum/optimal): 512 bytes / 512 bytes

Disk identifier: 0x35dd781b

              Device Boot      Start         End      Blocks   Id  System

/dev/mapper/mpathjlp1               1         130     1044193+  83  Linux

[root@rac1 ~]# chown oragrid:oinstall /dev/mapper/mpathjlp1

[root@rac2 ~]# chown oragrid:oinstall /dev/mapper/mpathjlp1

[root@rac1 ~]# ls -l /dev/dm-254

brw-rw---- 1 oragrid oinstall 252, 254 Nov 13 18:51 /dev/dm-254

[root@rac1 ~]# ls -l /dev/dm-254

brw-rw---- 1 oragrid oinstall 252, 254 Nov 13 18:51 /dev/dm-254

3.5) If second node does not see the disks you might need to use partprobe

E.g partprobe /dev/mapper/mpathjlp1

4) Create ASM disk

[root@rac1 ~]# /etc/init.d/oracleasm  createdisk  VOTEVPLEX /dev/mapper/mpathjlp1

Marking disk "VOTEVPLEX" as an ASM disk:                   [  OK  ]

[root@rac1 ~]# /etc/init.d/oracleasm  listdisks

[root@rac2 ~]# /etc/init.d/oracleasm  scandisks

Scanning the system for Oracle ASMLib disks:               [  OK  ]

[root@rac2 ~]# /etc/init.d/oracleasm  listdisks

5) Check ASM disk

[root@rac1 ~]# /etc/init.d/oracleasm querydisk -d VOTEVPLEX

Disk "VOTEVPLEX" is a valid ASM disk on device [252,254]

[root@rac2 ~]# /etc/init.d/oracleasm querydisk -d VOTEVPLEX

Disk "VOTEVPLEX" is a valid ASM disk on device [252,254]

[root@rac1 ~]# oracleasm querydisk /dev/dm-254

Device "/dev/dm-254" is marked an ASM disk with the label "VOTEVPLEX"

[root@rac2 ~]# oracleasm querydisk /dev/dm-254

Device "/dev/dm-254" is marked an ASM disk with the label "VOTEVPLEX"

[root@rac1 ~]# multipath -ll mpathjl

mpathjl (36000144000000010303f2aae3350e2e2) dm-253 XXX

size=1.0G features='1 queue_if_no_path' hwhandler='0' wp=rw

`-+- policy='round-robin 0' prio=1 status=active

  |- 1:0:6:106 sdob 128:368 active ready running

  |- 1:0:7:106 sdoc 128:384 active ready running

  |- 1:0:8:106 sdod 128:400 active ready running

  `- 1:0:9:106 sdoe 128:416 active ready running

[root@rac2 ~]# multipath -ll mpathjl

Nov 13 20:00:58 | multipath.conf line 4, invalid keyword: selector

Nov 13 20:00:58 | multipath.conf line 7, invalid keyword: prio_callout

mpathjl (36000144000000010303f2aae3350e2e2) dm-253 XXX

size=1.0G features='1 queue_if_no_path' hwhandler='0' wp=rw

`-+- policy='round-robin 0' prio=1 status=active

  |- 1:0:4:106 sdto 65:608  active ready running

  |- 2:0:8:106 sdts 65:672  active ready running

  |- 1:0:5:106 sdtp 65:624  active ready running

  |- 2:0:6:106 sdtt 65:688  active ready running

  |- 1:0:6:106 sdtr 65:656  active ready running

  |- 2:0:7:106 sdtu 65:704  active ready running

  `- 1:0:7:106 sdtq 65:640  active ready running

How to create static listener for Data Guard in Grid Infrastructure Standby environment

My environment has Grid Infrastructure installed and hence two separate homes. Please note that this server works as Data Guard node with the intention to be used as Active Data Guard -mode.

GRID_HOME = /oracle/app/11.2.0/grid
DB_HOME = /oracle/app/oracle/product/11.2.0/dbhome_1/

Primary Database = primsid
Primary SERVICE_NAME = PRIMARYSVC
Data Guard database instance name = DGSID
Data Guard database unique name = primsiddg
Data Guard SERVICE_NAME = DGSVC

What I wanted:

I wanted to create static listener for my Data Guard database, because I wanted to create the Data Guard -database from using the 'Active duplicate' -mode. This requires the standby database can be connected using listener connection even though the database is in NOMOUNT -state.

How to achieve this;

1) Do no create listener.ora in the GRID_HOME. Leave it empty.
2) Create listener.ora in the DB_HOME. These are my settings. Please note ORACLE_HOME and ADR_BASE_LISTENER - values.

SID_LIST_LISTENER =
  (SID_LIST =
    (SID_DESC =
      (GLOBAL_DBNAME = DGSVC)
      (ORACLE_HOME = /oracle/app/oracle/product/11.2.0/dbhome_1)
      (SID_NAME = DGSID)
    )
  )

LISTENER =
  (DESCRIPTION_LIST =
    (DESCRIPTION =
      (ADDRESS = (PROTOCOL = TCP)(HOST = dg-server.domain.com)(PORT = 1521))
    )
    (DESCRIPTION =
      (ADDRESS = (PROTOCOL = IPC)(KEY = EXTPROC1521))
    )
  )

ADR_BASE_LISTENER = /oracle/app/oracle/product/11.2.0/dbhome_1

3) Start the listener GRID_HOME with grid environment variables on

4) Check listener status

oracle@dg-server:/oracle/app/oracle/product/11.2.0/dbhome_1/network/admin$ listener status

LSNRCTL for Solaris: Version 11.2.0.4.0 - Production on 14-DEC-2014 17:34:58

Copyright (c) 1991, 2013, Oracle.  All rights reserved.

Connecting to (DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(HOST=dg-server.domain.com)(PORT=1521)))

STATUS of the LISTENER

------------------------

Alias                     LISTENER

Version                   TNSLSNR for Solaris: Version 11.2.0.4.0 - Production

Start Date                14-DEC-2014 17:33:42

Uptime                    0 days 0 hr. 1 min. 15 sec

Trace Level               off

Security                  ON: Local OS Authentication

SNMP                      ON

Listener Parameter File   /oracle/app/oracle/product/11.2.0/dbhome_1/network/admin/listener.ora

Listener Log File         /oracle/app/oracle/product/11.2.0/dbhome_1/diag/tnslsnr/dg-server/listener/alert/log.xml

Listening Endpoints Summary...

  (DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=dg-server)(PORT=1521)))

  (DESCRIPTION=(ADDRESS=(PROTOCOL=ipc)(KEY=EXTPROC1521)))

Services Summary...

Service "+ASM" has 1 instance(s).

  Instance "+ASM", status READY, has 1 handler(s) for this service...

Service "DGSVC" has 2 instance(s).

  Instance "DGSID", status UNKNOWN, has 1 handler(s) for this service...

  Instance "DGSID", status BLOCKED, has 1 handler(s) for this service...

The command completed successfully

In case you see double services like this "BLOCKED" -- check your local_listener -value at database and clear the value.

SQL> alter system set local_listener = '';

Jarjestelma on muutettu.

SQL> alter system register;


Jarjestelma on muutettu.

5) If you have instance with UNKNOWN -status, you're ok. UNKNOWN is the status which we want.
6) Start database with pfile on nomout -state
SQL> startup nomount pfile = 'initPRIMSID.ora';
ORACLE instance started.

Total System Global Area 6415736832 bytes
Fixed Size                  2239440 bytes
Variable Size            1136661552 bytes
Database Buffers         5272240128 bytes

Redo Buffers                4595712 bytes
7) Restore controlfiles
RMAN> restore controlfile from 'PRIMSID_stby.ctl';

8) Start active duplication
oracle@dg-server:/oracle/app/oracle/product/11.2.0/dbhome_1/network/admin$ rman TARGET sys/pw@PRIMARYSVC AUXILIARY sys/pw@DGSVC

Recovery Manager: Release 11.2.0.4.0 - Production on Sun Dec 14 17:46:34 2014

Copyright (c) 1982, 2011, Oracle and/or its affiliates.  All rights reserved.

connected to target database: PRIMSID (DBID=1185966409)
connected to auxiliary database: DGSID (not mounted)