lecture on database join (simplified)

Join (SQL)
--------------------

SQL JOIN clause combines records from two tables in a relational database, resulting in a new, temporary table, sometimes called a "joined table". A JOIN may also be thought of as a SQL operation that relates tables by means of values common between them. SQL specifies four types of JOIN:

1) INNER Join.
2) OUTER Join.
3) LEFT Join.
4) RIGHT Join.

In special cases, a table (base table, view, or joined table) can JOIN to itself in a self-join.

A programmer writes a JOIN predicate to identify the records for joining. If the predicate evaluates positively, the combined record is inserted into the temporary (or "joined") table. Any predicate supported by SQL can become a JOIN-predicate, for example, WHERE-clauses.

Mathematically, a JOIN provides the fundamental operation in relational algebra and generalizes function composition.

[edit] Sample tables
All subsequent explanations on join types in this article make use of the following two tables. The rows in these tables serve to illustrate the effect of different types of joins and join-predicates. In the following tables, Department.DepartmentID is the primary key, while Employee.DepartmentID is a foreign key.

Employee Table
LastName DepartmentID
Rafferty 31
Jones 33
Steinberg 33
Robinson 34
Smith 34
Jasper NULL
Department Table
DepartmentID DepartmentName
31 Sales
33 Engineering
34 Clerical
35 Marketing


Note: The "Marketing" Department currently has no listed employees. Employee "Jasper" has not been assigned to any Department yet.


Inner join
----------------

An inner join requires each record in the two joined tables to have a matching record.


An inner join essentially combines the records from two tables (A and B) based on a given join-predicate. The result of the join can be defined as the outcome of first taking the Cartesian product (or cross-join) of all records in the tables (combining every record in table A with every record in table B) - then return all records which satisfy the join predicate. Actual SQL implementations will normally use other approaches where possible, since computing the Cartesian product is not very efficient. This type of join occurs most commonly in applications, and represents the default join-type.

SQL specifies two different syntactical ways to express joins. The first, called "explicit join notation", uses the keyword JOIN, whereas the second uses the "implicit join notation". The implicit join notation lists the tables for joining in the FROM clause of a SELECT statement, using commas to separate them. Thus, it specifies a cross-join, and the WHERE clause may apply additional filter-predicates. Those filter-predicates function comparably to join-predicates in the explicit notation.

One can further classify inner joins as equi-joins, as natural joins, or as cross-joins (see below).

Programmers should take special care when joining tables on columns that can contain NULL values, since NULL will never match any other value (or even NULL itself), unless the join condition explicitly uses the IS NULL or IS NOT NULL predicates.

As an example, the following query takes all the records from the Employee table and finds the matching record(s) in the Department table, based on the join predicate. The join predicate compares the values in the DepartmentID column in both tables. If it finds no match (i.e., the department-id of an employee does not match the current department-id from the Department table), then the joined record remains outside the joined table, i.e., outside the (intermediate) result of the join.

Example of an explicit inner join:

SELECT *
FROM employee
INNER JOIN department
ON employee.DepartmentID = department.DepartmentID
Is equivalent to:

SELECT *
FROM employee, department
WHERE employee.DepartmentID = department.DepartmentID
Explicit Inner join result:

Employee.LastName Employee.DepartmentID Department.DepartmentName Department.DepartmentID
Smith 34 Clerical 34
Jones 33 Engineering 33
Robinson 34 Clerical 34
Steinberg 33 Engineering 33
Rafferty 31 Sales 31

Notice that the employee "Jasper" and the department "Marketing" do not appear. Neither of these has any matching records in the respective other table: "Jasper" has no associated department and no employee has the department ID 35. Thus, no information on Jasper or on Marketing appears in the joined table. Depending on the desired results, this behavior may be a subtle bug. Outer joins may be used to avoid it.


Types of inner joins
---------------------------------


Equi-join
-------------------
An equi-join, also known as an equijoin, is a specific type of comparator-based join, or theta join, that uses only equality comparisons in the join-predicate. Using other comparison operators (such as <) disqualifies a join as an equi-join. The query shown above has already provided an example of an equi-join:

SELECT Employee.lastName, Employee.DepartmentID, Department.DepartmentName
FROM Employee INNER JOIN Department
ON Employee.DepartmentID = Department.DepartmentID
ORDER BY Employee.lastName;
The resulting joined table contains two columns named DepartmentID, one from table Employee and one from table Department

SQL:2003 does not have a specific syntax to express equi-joins, but some database engines provide a shorthand syntax: for example, MySQL and PostgreSQL support USING(DepartmentID) in addition to the ON ... syntax.


Natural join
----------------------
A natural join offers a further specialization of equi-joins. The join predicate arises implicitly by comparing all columns in both tables that have the same column-name in the joined tables. The resulting joined table contains only one column for each pair of equally-named columns.

The above sample query for inner joins can be expressed as a natural join in the following way:

SELECT *
FROM employee NATURAL JOIN department
The result appears slightly different, however, because only one DepartmentID column occurs in the joined table.

DepartmentID Employee.LastName Department.DepartmentName
34 Smith Clerical
33 Jones Engineering
34 Robinson Clerical
33 Steinberg Engineering
31 Rafferty Sales
This section may contain original research or unverified claims.
Please help Wikipedia by adding references. See the talk page for details.(August 2008)

Using the NATURAL JOIN keyword to express joins can suffer from ambiguity at best, and could, in the case of poor coding or design, leave systems open to problems if schema changes occur in the database. For example, the removal, addition, or renaming of columns changes the semantics of a natural join. Thus, the safer approach involves explicitly coding the join-condition using a regular inner join, but such problems are likely to show in the cases of poor design.

The Oracle database implementation of SQL selects the appropriate column in the naturally-joined table from which to gather data. An error-message such as "ORA-25155: column used in NATURAL join cannot have qualifier" is an error to help prevent or reduce the problems that could occur may encourage checking and precise specification of the columns named in the query, and can also help in providing compile time checking (instead of errors in query).


Cross join
----------------
A cross join, cartesian join or product provides the foundation upon which all types of inner joins operate. A cross join returns the cartesian product of the sets of records from the two joined tables. Thus, it equates to an inner join where the join-condition always evaluates to True or join-condition is absent in statement.

If A and B are two sets, then the cross join is written as A × B.

The SQL code for a cross join lists the tables for joining (FROM), but does not include any filtering join-predicate.

Example of an explicit cross join:

SELECT *
FROM employee CROSS JOIN department
Example of an implicit cross join:

SELECT *
FROM employee, department;
Employee.LastName Employee.DepartmentID Department.DepartmentName Department.DepartmentID
Rafferty 31 Sales 31
Jones 33 Sales 31
Steinberg 33 Sales 31
Smith 34 Sales 31
Robinson 34 Sales 31
Jasper NULL Sales 31
Rafferty 31 Engineering 33
Jones 33 Engineering 33
Steinberg 33 Engineering 33
Smith 34 Engineering 33
Robinson 34 Engineering 33
Jasper NULL Engineering 33
Rafferty 31 Clerical 34
Jones 33 Clerical 34
Steinberg 33 Clerical 34
Smith 34 Clerical 34
Robinson 34 Clerical 34
Jasper NULL Clerical 34
Rafferty 31 Marketing 35
Jones 33 Marketing 35
Steinberg 33 Marketing 35
Smith 34 Marketing 35
Robinson 34 Marketing 35
Jasper NULL Marketing 35

The cross join does not apply any predicate to filter records from the joined table. Programmers can further filter the results of a cross join by using a WHERE clause.


[edit] Outer joins
An outer join does not require each record in the two joined tables to have a matching record. The joined table retains each record—even if no other matching record exists. Outer joins subdivide further into left outer joins, right outer joins, and full outer joins, depending on which table(s) one retains the rows from (left, right, or both).

(For a table to qualify as left or right its name has to appear after the FROM or JOIN keyword, respectively.)

No implicit join-notation for outer joins exists in SQL:2003.


[edit] Left outer join
The result of a left outer join (or simply left join) for tables A and B always contains all records of the "left" table (A), even if the join-condition does not find any matching record in the "right" table (B). This means that if the ON clause matches 0 (zero) records in B, the join will still return a row in the result—but with NULL in each column from B. This means that a left outer join returns all the values from the left table, plus matched values from the right table (or NULL in case of no matching join predicate).

For example, this allows us to find an employee's department, but still to show the employee even when their department does not exist (contrary to the inner-join example above, where employees in non-existent departments are excluded from the result).

Example of a left outer join, with the additional result row italicized:

SELECT *
FROM employee LEFT OUTER JOIN department
ON employee.DepartmentID = department.DepartmentID
Employee.LastName Employee.DepartmentID Department.DepartmentName Department.DepartmentID
Jones 33 Engineering 33
Rafferty 31 Sales 31
Robinson 34 Clerical 34
Smith 34 Clerical 34
Jasper NULL NULL NULL
Steinberg 33 Engineering 33


[edit] Right outer joins
A right outer join (or right join) closely resembles a left outer join, except with the tables reversed. Every record from the "right" table (B) will appear in the joined table at least once. If no matching row from the "left" table (A) exists, NULL will appear in columns from A for those records that have no match in A.

A right outer join returns all the values from the right table and matched values from the left table (NULL in case of no matching join predicate).

For example, this allows us to find each employee and their department, but still show departments that have no employees.

Example right outer join, with the additional result row italicized:

SELECT *
FROM employee RIGHT OUTER JOIN department
ON employee.DepartmentID = department.DepartmentID
Employee.LastName Employee.DepartmentID Department.DepartmentName Department.DepartmentID
Smith 34 Clerical 34
Jones 33 Engineering 33
Robinson 34 Clerical 34
Steinberg 33 Engineering 33
Rafferty 31 Sales 31
NULL NULL Marketing 35


[edit] Full outer join
A full outer join combines the results of both left and right outer joins. The joined table will contain all records from both tables, and fill in NULLs for missing matches on either side.

For example, this allows us to see each employee who is in a department and each department that has an employee, but also see each employee who is not part of a department and each department who doesn't have an employee.

Example full outer join:

SELECT *
FROM employee
FULL OUTER JOIN department
ON employee.DepartmentID = department.DepartmentID
Employee.LastName Employee.DepartmentID Department.DepartmentName Department.DepartmentID
Smith 34 Clerical 34
Jones 33 Engineering 33
Robinson 34 Clerical 34
Jasper NULL NULL NULL
Steinberg 33 Engineering 33
Rafferty 31 Sales 31
NULL NULL Marketing 35

Some database systems like db2 (version 2 and before) do not support this functionality directly, but they can emulate it through the use of left and right outer joins and unions. The same example can appear as follows:

SELECT *
FROM employee
LEFT JOIN department
ON employee.DepartmentID = department.DepartmentID
UNION
SELECT *
FROM employee
RIGHT JOIN department
ON employee.DepartmentID = department.DepartmentID
WHERE employee.DepartmentID IS NULL
or as follows:

SELECT *
FROM employee
LEFT JOIN department
ON employee.DepartmentID = department.DepartmentID
UNION
SELECT *
FROM department
LEFT JOIN employee
ON employee.DepartmentID = department.DepartmentID
WHERE employee.DepartmentID IS NULL
or as follows:

SELECT *
FROM department
RIGHT JOIN employee
ON employee.DepartmentID = department.DepartmentID
UNION
SELECT *
FROM employee
RIGHT JOIN department
ON employee.DepartmentID = department.DepartmentID
WHERE employee.DepartmentID IS NULL
Note: UNION=UNION ALL in above examples


[edit] Self-join
A self-join is joining a table to itself.[1] This is best illustrated by the following example.

Example

A query to find all pairings of two employees in the same country is desired. If you had two separate tables for employees and a query which requested employees in the first table having the same country as employees in the second table, you could use a normal join operation to find the answer table. However, all the employee information is contained within a single large table. [2]

Considering a modified Employee table such as the following:

Employee Table EmployeeID LastName Country DepartmentID
123 Rafferty Australia 31
124 Jones Australia 33
145 Steinberg Australia 33
201 Robinson United States 34
305 Smith United Kingdom 34
306 Jasper United Kingdom NULL



An example solution query could be as follows:

SELECT F.EmployeeID, F.LastName, S.EmployeeID, S.LastName, F.Country
FROM Employee F, Employee S
WHERE F.Country = S.Country
AND F.EmployeeID < S.EmployeeID
ORDER BY F.EmployeeID, S.EmployeeID;
Which results in the following table being generated.

Employee Table after Self-join by Country EmployeeID LastName EmployeeID LastName Country
123 Rafferty 124 Jones Australia
123 Rafferty 145 Steinberg Australia
124 Jones 145 Steinberg Australia
305 Smith 306 Jasper United Kingdom

For this example, note that:

F and S are aliases for the first and second copies of the employee table.
The condition F.Country = S.Country excludes pairings between employees in different countries. The example question only wanted pairs of employees in the same country.
The condition F.EmployeeID < S.EmployeeID excludes pairings where the EmployeeIDs are the same.
F.EmployeeID < S.EmployeeID also excludes duplicate pairings. Without it only the following less useful part of the table would be generated (for the United Kingdom only shown):
EmployeeID LastName EmployeeID LastName Country
305 Smith 305 Smith United Kingdom
305 Smith 306 Jasper United Kingdom
306 Jasper 305 Smith United Kingdom
306 Jasper 306 Jasper United Kingdom

Only one of the two middle pairings is needed to satisfy the original question, and the topmost and bottommost are of no interest at all in this example.


[edit] Alternatives
The effect of outer joins can also be obtained using correlated subqueries. For example

SELECT employee.LastName, employee.DepartmentID, department.DepartmentName
FROM employee LEFT OUTER JOIN department
ON employee.DepartmentID = department.DepartmentID
can also be written as

SELECT employee.LastName, employee.DepartmentID,
(SELECT department.DepartmentName
FROM department
WHERE employee.DepartmentID = department.DepartmentID )
FROM employee...........

[edit] Implementation
Much work in database-systems has aimed at efficient implementation of joins, because relational systems commonly call for joins, yet face difficulties in optimising their efficient execution. The problem arises because (inner) joins operate both commutatively and associatively. In practice, this means that the user merely supplies the list of tables for joining and the join conditions to use, and the database system has the task of determining the most efficient way to perform the operation. A query optimizer determines how to execute a query containing joins. A query optimizer has two basic freedoms:

Join order: Because joins function commutatively and associatively, the order in which the system joins tables does not change the final result-set of the query. However, join-order does have an enormous impact on the cost of the join operation, so choosing the best join order becomes very important.
Join method: Given two tables and a join condition, multiple algorithms can produce the result-set of the join. Which algorithm runs most efficiently depends on the sizes of the input tables, the number of rows from each table that match the join condition, and the operations required by the rest of the query.
Many join-algorithms treat their inputs differently. One can refer to the inputs to a join as the "outer" and "inner" join operands, or "left" and "right", respectively. In the case of nested loops, for example, the database system will scan the entire inner relation for each row of the outer relation.

One can classify query-plans involving joins as follows:

left-deep
using a base table (rather than another join) as the inner operand of each join in the plan
right-deep
using a base table as the outer operand of each join in the plan
bushy
neither left-deep nor right-deep; both inputs to a join may themselves result from joins
These names derive from the appearance of the query plan if drawn as a tree, with the outer join relation on the left and the inner relation on the right (as convention dictates).


[edit] Join algorithms
Three fundamental algorithms exist for performing a join operation.


[edit] Nested loops
Please refer to main articles: Nested loop join and block nested loop
Use of nested loops produces the simplest join-algorithm. For each tuple in the outer join relation, the system scans the entire inner-join relation and appends any tuples that match the join-condition to the result set. Naturally, this algorithm performs poorly with large join-relations: inner or outer or both. An index on columns in the inner relation in the join-predicate can enhance performance.

The block nested loops (BNL) approach offers a refinement to this technique: for every block in the outer relation, the system scans the entire inner relation. For each match between the current inner tuple and one of the tuples in the current block of the outer relation, the system adds a tuple to the join result-set. This variant means doing more computation for each tuple of the inner relation, but far fewer scans of the inner relation.


Merge join
-------------------
If both join relations come in order, sorted by the join attribute(s), the system can perform the join trivially, thus:

Consider the current "group" of tuples from the inner relation; a group consists of a set of contiguous tuples in the inner relation with the same value in the join attribute.
For each matching tuple in the current inner group, add a tuple to the join result. Once the inner group has been exhausted, advance both the inner and outer scans to the next group.
Merge joins offer one reason why many optimizers keep track of the sort order produced by query plan operators—if one or both input relations to a merge join arrives already sorted on the join attribute, the system need not perform an additional sort. Otherwise, the DBMS will need to perform the sort, usually using an external sort to avoid consuming too much memory.

WAP AND WML SIMPLIFIED

The WAP protocol was designed to show internet contents on wireless clients, like mobile phones.
------------------------------------------------------------
What You Should Already Know
Before you continue you should have a basic understanding of the following:
HTML
JavaScript
XML
--------------------------------------------------------------------------------

What is WAP?
The wireless industry came up with the idea of WAP. The point of this standard was to show internet contents on wireless clients, like mobile phones.

WAP stands for Wireless Application Protocol
WAP is an application communication protocol
WAP is used to access services and information
WAP is inherited from Internet standards
WAP is for handheld devices such as mobile phones
WAP is a protocol designed for micro browsers
WAP enables the creating of web applications for mobile devices.
WAP uses the mark-up language WML (not HTML)
WML is defined as an XML 1.0 application

--------------------------------------------------------------------------------

The Wireless Application Protocol
The WAP protocol is the leading standard for information services on wireless terminals like digital mobile phones.

The WAP standard is based on Internet standards (HTML, XML and TCP/IP). It consists of a WML language specification, a WMLScript specification, and a Wireless Telephony Application Interface (WTAI) specification.

WAP is published by the WAP Forum, founded in 1997 by Ericsson, Motorola, Nokia, and Unwired Planet. Forum members now represent over 90% of the global handset market, as well as leading infrastructure providers, software developers and other organizations. You can read more about the WAP forum at our WAP Forum page.


--------------------------------------------------------------------------------

WAP Micro Browsers
To fit into a small wireless terminal, WAP uses a Micro Browser.

A Micro Browser is a small piece of software that makes minimal demands on hardware, memory and CPU. It can display information written in a restricted mark-up language called WML.

The Micro Browser can also interpret a reduced version of JavaScript called WMLScript.


--------------------------------------------------------------------------------

What is WML?
WML stands for Wireless Markup Language. It is a mark-up language inherited from HTML, but WML is based on XML, so it is much stricter than HTML.

WML is used to create pages that can be displayed in a WAP browser. Pages in WML are called DECKS. Decks are constructed as a set of CARDS.


--------------------------------------------------------------------------------

What is WMLScript?
WML uses WMLScript to run simple code on the client. WMLScript is a light JavaScript language. However, WML scripts are not embedded in the WML pages. WML pages only contains references to script URLs. WML scripts need to be compiled into byte code on a server before they can run in a WAP browser.

Visit our WMLScript tutorial to learn more about scripting in WML documents.


--------------------------------------------------------------------------------

Examples of WAP use
Checking train table information
Ticket purchase
Flight check in
Viewing traffic information
Checking weather conditions
Looking up stock values
Looking up phone numbers
Looking up addresses
Looking up sport results

--------------------------------------------------------------------------------


WAP Simplification
------------------------------

WAP is an open international standard for application layer network communications in a wireless communication environment. Its main use is to enable access to the Internet (HTTP) from a mobile phone or PDA.

A WAP browser provides all of the basic services of a computer based web browser but simplified to operate within the restrictions of a mobile phone, such as its smaller view screen. WAP sites are websites written in, or dynamically converted to, WML (Wireless Markup Language) and accessed via the WAP browser.

Before the introduction of WAP, service providers had extremely limited opportunities to offer interactive data services. Interactive data applications are required to support now commonplace activities such as:

Email by mobile phone
Tracking of stock market prices
Sports results
News headlines
Music downloads



Another lesson on WAP
------------------------------------
The original WAP was a simple platform for access to web-like WML services and e-mail using mobile phones in Europe and the SE Asian regions and continues today with a considerable user base. The later versions of WAP were primarily for the United States region and was designed for a different requirement - to enable full web XHTML access using mobile devices with a higher specification and cost, and with a higher degree of software complexity.

There has been considerable discussion about whether the WAP protocol design was appropriate. Some have suggested that the bandwidth-sparing simple interface of Gopher would be a better match for mobile phones and Personal digital assistants (PDAs).

The initial design of WAP was specifically aimed at protocol independence across a range of different protocols (SMS, IP over PPP over a circuit switched bearer, IP over GPRS, etc). This has led to a protocol considerably more complex than an approach directly over IP might have caused.

Most controversial, especially for many from the IP side, was the design of WAP over IP. WAP's transmission layer protocol, WTP, uses its own retransmission mechanisms over UDP to attempt to solve the problem of inadequacy using TCP over high packet loss networks.



WML BASICS
---------------
Wireless Markup Language
----------------------------------------

Evolution of mobile web standardsWireless Markup Language, based on XML, is a markup language intended for devices that implement the Wireless Application Protocol (WAP) specification, such as mobile phones, and preceded the use of other markup languages now used with WAP, such as XHTML and even standard HTML (which are gaining in popularity as processing power in mobile devices increases).

Contents
1 WML history
2 WML markup
3 Criticism
4 References
5 See also
6 External links



WML history
Building on Openwave's HDML, Nokia's "Tagged Text Markup Language" (TTML) and Ericsson's proprietary markup language for mobile content, the WAP Forum created the WML 1.1 standard in 1998[1]. WML 2.0 was specified in 2001[2] , but has not been widely adopted. It was an attempt at bridging WML and XHTML Basic before the WAP 2.0 spec was finalized [3]. In the end, XHTML Mobile Profile became the markup language used in WAP 2.0. The newest WML version in active use is 1.3.


WML markup
WML documents are XML documents that validate against the WML DTD (Document Type Definition)[4] . The W3C Markup Validation service (http://validator.w3.org/) can be used to validate WML documents (they are validated against their declared document type).
Wireless Markup Language is a lot like HTML (Hyper Text Markup Language) in that it provides navigational support, data input, hyperlinks, text and image presentation, and forms. A WML document is known as a “deck”. Data in the deck is structured into one or more “cards” (pages) – each of which represents a single interaction with the user. The introduction of the terms "deck" and "card" into the internet and mobile phone communities was a result of the user interface software and its interaction with wireless communications services having to comply with the requirements of the laws of two or more nations.[citation needed]

WML decks are stored on an ordinary web server trivially configured to serve the text/vnd.wap.wml MIME type in addition to plain HTML and variants. The WML cards when requested by a device are accessed by a bridge WAP gateway, which sits between mobile devices and the World Wide Web, passing pages from one to the other much like a proxy. The gateways send the WML pages on in a form suitable for mobile device reception (WAP Binary XML). This process is hidden from the phone, so it may access the page in the same way as a browser accesses HTML, using a URL (for example, http://example.com/foo.wml). (Provided the mobile phone operator has not specifically locked the phone to prevent access of user-specified URLs.)

WML has a scaled down set of procedural elements which can be used by the author to control navigation to other cards.

It is an error and misconception to think of WML as a pinhole view of the Internet. The real power and value of WML is that it provides an interface for the phone hardware to initiate a call based on web content requested by user query. Consider a service that lets you enter a zip code, and obtain a list of clickable phone numbers of pizza parlors and taxicabs in your immediate location:

Mobile devices are moving towards support for greater amounts of XHTML and even standard HTML as processing power in handsets increases. These standards are concerned with formatting and presentation. They do not however address cell-phone or mobile device hardware interfacing in the same way as WML.