I had the same problem.but my tables were small like 300,000 rows.
I moved those tables to a same database.In you case it may be DB1 or DB2.Then made the join.It was fast enough and very end of process Iam dropping those temporary table.So the data base performance does not effect…
Yes I’m am looking for any job that doesn’t exists in table 1, and I tried your suggestion as follows:
(STAGE_FACTGLTRANS_TMP.GLENTITY = SF_ENTITY_TMP.GLENTITY__C) AND
(STAGE_FACTGLTRANS_TMP.JOBNO = SF_CAPEX_C_TMP.PROJECT_ID_JOB_NUMBER__C) AND
(SF_CAPEX_C_TMP.PROJECT_ID_JOB_NUMBER__C IS NULL )
Where I set SF_CAPEX_C_TMP as the outer source in the OUTER Join tab and STAGE_FACTGLTRANS_TMP as the inner source.
But Now I recieve the followign error when attemptiong to validate:
[Query:Query]
BODI-1111077: Invalid query <Inner table <STAGE_FACTGLTRANS_TMP> of an outer join is being joined with table <SF_ENTITY_TMP>. Inner tables of an outer join cannot be involved in an inner join.>.
Any ideas on what is wrong here? Thanks again for you help.
When I switch up the table sin the Outer Join tab I recieve the following errors
[Query:Query]
BODI-1111077: Invalid query <Unsupported combination of inner and outer joins. Table <SF_ENTITY_TMP> is being inner joined in the query transform that contains outer join(s). Please split the query transform into separate queries for the inner and outer joins.>.
I tested it and I got it working. DI dont like inner and outer joins mixed you have to make all outer joins…
i.e.
I created 3 tables with 1 column only
X, XX and XXX with respectively columns “1”, “2” and “3”
I filled X with values a, b and c
XX with a and b
XXX only with a.
Now in DI, join the tables as you are used to… :
X.1 = XX.2
XX.2 = XXX.3
Now if you want to outer join XXX only you cannot do that. You also have to outerjoin XX to X.
So in the outerjoin tab you have to create 2 rows.
X - XX
XX - XXX
In order to “force” the innerjoin between X - XX you have to add something to your where… :
XX.2 is not null
Now the result should look like:
a - a - a
b - b - null
i.e. a single outerjoin to XXX. (missing the c - null - null option)
I hope this is not as clear as mud and you can make some sence of my example…
Thanks Erik, maybe I didn’t fully understand your explanation. I know you spelled it out for me, but can you align it to my exact case. Thanks for your patience.
Doing an inner join where table1.field1 <> table2.field2 is very different to outer join table1.field1 = table2.field2. This first is matching table1 with every record in table2 that does not have a matching field2 value, whereas the second is only finding a match if there is a matching field2 value…and you’d add a second query to then filter out the ones where table2.field2 is null (it did not find a match)
ugh offcourse that should read:
and xx.entity_no is not null
This should be the “proper” sql, tho that doesnt work in DI for some reason.
FROM STAGE_FACTGLTRANS_TMP
join SF_CAPEX_C_TMP on STAGE_FACTGLTRANS_TMP.JOBNO = SF_CAPEX_C_TMP.PROJECT_ID_JOB_NUMBER__C
left join SF_ENTITY_TMP on SF_CAPEX_C_TMP.ENTITY__C = SF_ENTITY_TMP.ID
where SF_ENTITY_TMP.ID is null
This is what the DI query would look like:
FROM STAGE_FACTGLTRANS_TMP
left join SF_CAPEX_C_TMP on STAGE_FACTGLTRANS_TMP.JOBNO = SF_CAPEX_C_TMP.PROJECT_ID_JOB_NUMBER__C
left join SF_ENTITY_TMP on SF_CAPEX_C_TMP.ENTITY__C = SF_ENTITY_TMP.ID
where SF_ENTITY_TMP.ID is null
and SF_CAPEX_C_TMP.PROJECT_ID_JOB_NUMBER__C is not null
Or you can re-write that to:
FROM STAGE_FACTGLTRANS_TMP
, SF_CAPEX_C_TMP
, SF_ENTITY_TMP
where STAGE_FACTGLTRANS_TMP.JOBNO = SF_CAPEX_C_TMP.PROJECT_ID_JOB_NUMBER__C
and SF_CAPEX_C_TMP.PROJECT_ID_JOB_NUMBER__C is not null
and SF_CAPEX_C_TMP.ENTITY__C = SF_ENTITY_TMP.ID
and SF_ENTITY_TMP.ID is null
Now above where-clause you can copy paste into DI and in the Outer join you should put 2 rows
Outer join - inner join
STAGE_FACTGLTRANS_TMP - SF_CAPEX_C_TMP
SF_CAPEX_C_TMP - SF_ENTITY_TMP
You lost me tho… you want everything that doesnt match in STAGE_FACTGLTRANS_TMP but you are selecting data from that same table which I think will only return empty info?
STAGE_FACTGLTRANS_TMP.JOBNO
1
3
4
5
SF_CAPEX_C_TMP.PROJECT_ID_JOB_NUMBER__C SF_CAPEX_C_TMP.ENTITY__C
1 a
2 b
3 c
4 d
5 e
6 f
SF_ENTITY_TMP.ID
a
b
c
d
e
f
g
Given above data for the tables, you want to see:
b - 2 -
f - 6 -
Right? Atleast that is what I get from you… but I may be wrong.
As a general remark STAGE = TMP generaly so for one I dont get the added TMP part
select table3.Job, Table3.entity
from table3
join table2 on table3.entity = table2.entity
left join table1 on table3.Job = Table1.job
where table1.job is null
For DI this would be there where:
table1.job is null
and table2.entity is not null
and make 2 rows in Outer join
Table3 - Table2
Table3 - Table1
Effectively making this query:
select table3.Job, Table3.entity
from table3
left join table2 on table3.entity = table2.entity
left join table1 on table3.Job = Table1.job
where table1.job is null
and table2.entity is not null
DI can not cope with an outer join and a filter to restrict WHERE Field IS NULL in the SAME QUERY. You need to split this into 2 querys to achieve.
In Query1 put the two tables to perform the outer join, put in the matching fields in the where clause.
In Query2 put in the 3rd table to inner join, in the where clause put in the join matching fields and there you can put in the field from the 1st (outer) join to filter out the non-matches (where field IS NULL).
As posted earlier, this is mentioned in the other thread
ugh you are not kidding… I just tested it and you are right
There is however no need to do this in 2 seperate queries… DI will perform the second join in memory (IIRC) which may be very slow.
Just make 2 outer-joins then enforce the inner join by doing “is not null” like I suggested. But appearently you need 2 queries to do “is null” and “is not null” ??