References
Creating Required Directories for Oracle Database Files on Shared File Systems .
Overview
The
EXAMPLE
database successfully created on the
PENRITH
cluster after I corrected the file and directory permissions for the ASM files to allow the
oracle
user read/write permission on those files.
Corrections have been made based on information in 14 Set Up DirectNFS for Example Database .
Analysis
The analysis put forward in
12 Remove DirectNFS in ASM
was flawed because I should have realised that ASM itself was able to successfully write to the
/u07
files because ASM was able to initialise and maintain its metadata there.
By reviewing
Creating Required Directories for Oracle Database Files on Shared File Systems
, I suspected that the security settings on the
/u07
files were not correct. I was able to confirm this as follows:
[oracle@penrith1 ~]$ ls -ld /u07 drwxr-xr-x 2 asm asmdba 4 Jan 4 23:04 /u07 [oracle@penrith1 ~]$ sudo su - [root@penrith1 ~]# ls -l /u07 total 7340190 -rw-r--r-- 1 asm asmdba 3255435264 Jan 8 11:50 data.dbf -rw-r--r-- 1 asm asmdba 4259840000 Jan 8 11:50 recovery.dbf
The
oracle
user does
not
have write access to these files and this is a much simpler explanation of the
ORA-15080: synchronous I/O operation to a disk failed
message.
Procedure
Change Permissions
Following
Creating Required Directories for Oracle Database Files on Shared File Systems
, I chose to use the
oinstall
group to allow the
oracle
user read/write access to the ASM files. I issued the following commands:
[root@penrith1 ~]# chmod 775 /u07 [root@penrith1 ~]# chown -R asm:oinstall /u07 [root@penrith1 ~]# chmod 664 /u07/*.dbf [root@penrith1 ~]# ls -ld /u07 drwxrwxr-x 2 asm oinstall 4 Jan 4 23:04 /u07 [root@penrith1 ~]# ls -l /u07 total 7340190 -rw-rw-r-- 1 asm oinstall 3255435264 Jan 8 11:51 data.dbf -rw-rw-r-- 1 asm oinstall 4259840000 Jan 8 11:51 recovery.dbf
Check ASM Access to Files
I restarted ASM on
PENRITH1
in order to confirm that there were no ill effects:
[asm@penrith1 ~]$ srvctl stop asm -n penrith1 [asm@penrith1 ~]$ srvctl start asm -n penrith1 [asm@penrith1 ~]$ asmcmd lsdg State Type Rebal Sector Block AU Total_MB Free_MB Req_mir_free_MB Usable_file_MB Offline_disks Name MOUNTED EXTERN N 512 4096 1048576 3104 3010 0 3010 0 DATA/ MOUNTED EXTERN N 512 4096 1048576 4062 3969 0 3969 0 RECOVERY/
Create Database
The
dbca
command was able to complete successfully.
Enable DirectNFS for ASM
I enabled DirectNFS for ASM as follows, by following the procedure in 10 Configure ASM Storage ,:
[asm@penrith1 dbs]$ cd /u01/app/asm/product/11.1.0/db_1/dbs [asm@penrith1 dbs]$ cat >oranfstab <<DONE > server: freenas > path: 192.168.2.1 > local: 192.168.2.2 > export: /mnt/PENRITH/ASMDATA > mount: /u07 > DONE [asm@penrith1 dbs]$ cat oranfstab server: freenas path: 192.168.2.1 local: 192.168.2.2 export: /mnt/PENRITH/ASMDATA mount: /u07 [asm@penrith1 dbs]$ srvctl stop database -d example [asm@penrith1 dbs]$ srvctl stop asm -n penrith1 [asm@penrith1 dbs]$ cd ../lib [asm@penrith1 lib]$ cp libodm11.so libodm11.so_stub [asm@penrith1 lib]$ ln -s libnfsodm11.so libodm11.so ln: creating symbolic link `libodm11.so' to `libnfsodm11.so': File exists [asm@penrith1 lib]$ ls -l libodm11.so lrwxrwxrwx 1 oracle oinstall 14 Jan 8 21:44 libodm11.so -> libnfsodm11.so [asm@penrith1 lib]$ srvctl start asm -n penrith1 [asm@penrith1 lib]$ srvctl start instance -d example -i example1
And the
EXAMPLE
database was successfully opened.
DirectNFS Information
The DirectNFS information is available on the ASM instance as follows:
[asm@penrith1 ~]$ sqlplus / as sysdba
SQL*Plus: Release 11.1.0.6.0 - Production on Sun Jan 8 15:25:10 2012
Copyright (c) 1982, 2007, Oracle. All rights reserved.
Connected to:
Oracle Database 11g Enterprise Edition Release 11.1.0.6.0 - Production
With the Partitioning, Real Application Clusters, OLAP, Data Mining
and Real Application Testing options
SQL> select * from v$dnfs_servers;
ID SVRNAME DIRNAME MNTPORT NFSPORT WTMAX RTMAX
---------- ------------ -------------------- ---------- ---------- ---------- ----------
1 freenas /mnt/PENRITH/ASMDATA 623 2049 32768 32768
Conclusion
Faulty analysis was done on my part (very passive voice here!). The file permissions was the core issue.
DirectNFS is now being used after the corrections from 14 Set Up DirectNFS for Example Database were applied.